{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 5.5 实践：基于ResNet18网络完成图像分类任务\n",
    "\n",
    "在本实践中，我们实践一个更通用的图像分类任务。\n",
    "\n",
    "**图像分类**（Image Classification）是计算机视觉中的一个基础任务，将图像的语义将不同图像划分到不同类别。很多任务也可以转换为图像分类任务。比如人脸检测就是判断一个区域内是否有人脸，可以看作一个二分类的图像分类任务。\n",
    "\n",
    "这里，我们使用的计算机视觉领域的经典数据集：CIFAR-10数据集，网络为ResNet18模型，损失函数为交叉熵损失，优化器为Adam优化器，评价指标为准确率。\n",
    "\n",
    "*******\n",
    "\n",
    "Adam优化器的介绍参考《神经网络与深度学习》第7.2.4.3节。\n",
    "\n",
    "*******"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.5.1 数据处理\n",
    "\n",
    "#### 5.5.1.1 数据集介绍\n",
    "\n",
    "CIFAR-10数据集包含了10种不同的类别、共60,000张图像，其中每个类别的图像都是6000张，图像大小均为$32\\times32$像素。CIFAR-10数据集的示例如 **图5.15** 所示。\n",
    "\n",
    "<center><img src=\"https://ai-studio-static-online.cdn.bcebos.com/12cb79a15a3d40fa9b0124dfe4681ab8b60b82d067fb433eb6da600b7c2ec875\" width=\"600\" hegiht=\"\" ></center>\n",
    "<center><br>图5.15：CIFAR-10数据集示例</br></center>\n",
    "<br></br>\n",
    "\n",
    "将数据集文件进行解压："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cifar-10-batches-py/\n",
      "cifar-10-batches-py/data_batch_4\n",
      "cifar-10-batches-py/readme.html\n",
      "cifar-10-batches-py/test_batch\n",
      "cifar-10-batches-py/data_batch_3\n",
      "cifar-10-batches-py/batches.meta\n",
      "cifar-10-batches-py/data_batch_2\n",
      "cifar-10-batches-py/data_batch_5\n",
      "cifar-10-batches-py/data_batch_1\n"
     ]
    }
   ],
   "source": [
    "# 解压数据集\n",
    "# 初次运行时将注释取消，以便解压文件\n",
    "# 如果已经解压过，不需要运行此段代码，否则由于文件已经存在，解压时会报错\n",
    "!tar -xvf /home/aistudio/data/data9154/cifar-10-python.tar.gz -C /home/aistudio/datasets/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 5.5.1.2 数据读取\n",
    "\n",
    "在本实验中，将原始训练集拆分成了train_set、dev_set两个部分，分别包括40 000条和10 000条样本。将data_batch_1到data_batch_4作为训练集，data_batch_5作为验证集，test_batch作为测试集。\n",
    "最终的数据集构成为：\n",
    "\n",
    "* 训练集：40 000条样本。\n",
    "* 验证集：10 000条样本。\n",
    "* 测试集：10 000条样本。\n",
    "\n",
    "读取一个batch数据的代码如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import pickle\n",
    "import numpy as np\n",
    "\n",
    "def load_cifar10_batch(folder_path, batch_id=1, mode='train'):\n",
    "    if mode == 'test':\n",
    "        file_path = os.path.join(folder_path, 'test_batch')\n",
    "    else:\n",
    "        file_path = os.path.join(folder_path, 'data_batch_'+str(batch_id))\n",
    "\n",
    "    #加载数据集文件\n",
    "    with open(file_path, 'rb') as batch_file:\n",
    "        batch = pickle.load(batch_file, encoding = 'latin1')\n",
    "\n",
    "    imgs = batch['data'].reshape((len(batch['data']),3,32,32)) / 255.\n",
    "    labels = batch['labels']\n",
    "\n",
    "    return np.array(imgs, dtype='float32'), np.array(labels)\n",
    "\n",
    "imgs_batch, labels_batch = load_cifar10_batch(folder_path='datasets/cifar-10-batches-py', \n",
    "                                                batch_id=1, mode='train')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "查看数据的维度："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "batch of imgs shape:  (10000, 3, 32, 32) batch of labels shape:  (10000,)\n"
     ]
    }
   ],
   "source": [
    "#打印一下每个batch中X和y的维度\n",
    "print (\"batch of imgs shape: \",imgs_batch.shape, \"batch of labels shape: \", labels_batch.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可视化观察其中的一张样本图像和对应的标签，代码如下所示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The label in the picture is 9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJIAAACPCAYAAAARM4LLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAF95JREFUeJztXVtsnNdx/mbvu+RyeadIihIpilIkS7aVKIpvjYMkjp00hVMkDeICbQoE8EuKtmgfGqQvLdAC6UvapxYw0KAuWtQxkDQxnASJajuJU8cX2bJ1F0XdKZEURXK5S+599/RhN//MnFjSSvq9kqjzAYJnOWf//9/12TP3GTLGwMHhZhG41Q/gsDbgNpKDL3AbycEXuI3k4AvcRnLwBW4jOfgCt5EcfMFNbSQieoKIjhPRFBF9w6+HcrjzQDfqkCSiIIBJAI8BmAbwFoCnjDFH/Hs8hzsFoZt47x4AU8aYUwBARM8BeBLAFTdSsiNlevoHAAClQk7xKqWCRxtDiheOxDw6EmU6GI6odYEAv6+QX1G8UjHP169WPZqg7xUIBpkX0Ad2W3vSo6PiOUy1otbl8/Kz6R9qzdTEM+YVryquI3/g9m+9UuFr1GqaacT1Q6GQoIN6HariPfr6Nb4EltOZy8aYPlwDN7ORhgGcF6+nAXzsam/o6R/A33z7X+qLj72tePOnj3p0taofa2DDhzx6w/g2j+5at0Gti8X5fZOHX1O8s1MHPLqc5U0WtO7V0ZXy6FAsoXh7Hv64R2/ews9UWF5U6w4f2u/RtVpJ8Upl/sEcOXxQ8TLpyx5dLBX5eUt6Eywu8EZdyRUUr1Ll9/X1dXt0V3e7Wlc1WX5PWbFQyPPO+uH//OwsmsAHrmwT0dNEtI+I9mUzyx/07RxuEW7mRLoAYES8Xt/4m4Ix5hkAzwDAyNi4ySzVf709nd16Xd8A06EOxRvcsMmjqzX++QRqWjzWciwaCksL+vp5/uUO9/Z79IaRzWrdyOaNHj00vF7x+vv5GcPhqEdXOvXJNbJ+HfMq+kQqFFicpZe0+L18mU+2kBDnIH0idfXwvWNtWjwuZ5Y8Ohrj/701o8VvOMTXyCynFa9UvH69+WZOpLcATBDRGBFFAHwFwAs3cT2HOxg3fCIZYypE9KcAfgogCOA7xpjDvj2Zwx2FmxFtMMb8GMCPfXoWhzsYN7WRrhvGAOW6jlMqalMhl2NdYnTLsOKtrK56tLR6untTal0ozJJ6YmKL4j30wG6PHh5g3SeV0pZtOcRmcSIWVbyQUB2owjpHflXrOsUyf7ZEXOtPXZ2sn41v2q54R48eFzfgaxSLWhdMdXR5tOUBwXJmzqMN+Du13QRLS/yd5nNFxbsR16ILkTj4AreRHHxBS0WbqdVQaZi/VKkqXjQS9+jly5cVr2cdi6IN97C53j8ypNaF5TlvednKFRaJx2bYNZA7Na/XBVgcHD/4nuJ9dBuLoo/v+ahH22GmjPCXnTt7UfEiYeGlj2g3R28fi/Rz50/wOssxupJnsZTJ6O8qFGZPfUcHv0972wHpjJeecgCIRi152QTcieTgC9xGcvAFbiM5+IKW60jFXF2+t8djitfRzWb4h++7X/FGNk14dFaY3cdPnVfrMjkRzExrt/9CmvWimVkOI3RY5j8CbAq/+N3vKVb4y/y7e/TBR/jvYa2PrVsndDejdZj0EgdL39l/QPFCIuzSlmT9qVLVOlhphT9b0DoKZKC2WmV9b2FRP0cArD/JLAEA6OzUbpVm4E4kB1/gNpKDL2ipaKMAIRoNAwDKwaTi5eOcL3M6oyPa7/7qTY9eXGAv8oWLc2pdOMimbzigTdqiiMIXCkwP9umv4NIsp990WGZwNp3x6MnTp/kag736OcJ8zcGRdYo3JF6fm9Wi+fhBft0/yCL3zDktllAWiW0l/TmrwjMfi7CojIbCal2+wOs6OrQbIhTSHv1m4E4kB1/gNpKDL2ipaAsEQkgk6slhl9I60WrqPB/rRw4f0u8ToqIqgr357KpaFxTiLF/MKF46y6+zIsh6ZvqoWtcWZ5G7dXyr/gBCPP7fqz/36I1jY2rZlq0cMO7p0RaQTDZLdWgREqiwR3y1yL9xO6iaT7PlV63qVNtYnEXYSobXdSS1+IrGOFmuVLID6NoL3gzcieTgC9xGcvAFbiM5+IKW6kjBYAid3XVTeer8pOLNnGFzOhHWOsHyKnuiVzKXPJpq2vRNizKjdF7rDqEo6w69A5xcFk9qHWZ49D6PHonppPvT7/2aPwuxvlSu6kyG+cvsRd+5c5vibZ7gQoaRQe1Vb39gl0cfOHbOo4sFHQUohoX5D637yCT/2VnOPIhEtT6W6uoXr7Sumbfq7ZqBO5EcfIHbSA6+oKWirVhcxcmTdS/1sZNTindx5qRHVy2zPplq8+itE6MevWPbDrVuZp6P5LPz+hp967gmbeM4m+vJnn61bk7kMpvLpxXv3FkWN/MiCLxNp17jsS0szlZXtJioCSloSrrm7fDrLDontnLgemC4U617/c1fevTsnHZzlMuiti/P118SwWIAiLfzNWUZOQCs5vR31wzcieTgC9xGcvAFbiM5+IKW6kirKxm8/su99RsP6PDD+LadHh23ItrbtnNi29YtXAhQLVitWgKsj6zCTooX7XCCrB+UK9osXs1y/X2qpMM4MsHs3CV2ScTadcsDWXe2aXxUP6P47ebTOhRx7I13eV2ev4Mdjz+h1u28l10I+X1aRzo5dcajEwnOqEh19kCDlbWM6BcA/HYdXTO45olERN8hoktEdEj8rZuI9hLRicZ/u652DYe1j2ZE278DeML62zcAvGSMmQDwUuO1w12Ma4o2Y8wviWjU+vOTAD7RoJ8F8HMAf32ta5VLFVw6Xxc5u+77XcWLRtnL260lFgaH2Hu7KCLf56d0g6tSjcVUgLS3ORhiUVE1wnNe0V9BVXV20yK2PcUJbAsrbCIHIm1qXU3Vudnt0MT1YtorPTrEXYJiQX5fALokfOcOdl90dmrXwAv5n3n07AyLrOF+XQNYJfb8y0Q8AMhkpLjU2RFXwo0q2wPGmJkGPQtg4GqLHdY+blrZNsYYIrpi2wEiehrA0wAQDoevtMzhDseNbqQ5Iho0xswQ0SCAS1daKDu2tbd3mER7vVwmbG29dJovEe3Wx3VOlBQXRCw23qXzvqM10Vi0oEWbEZ+0UGarRPadBICACMbWAprX3sPiIWJYrAbj2tYwEZbNNdIWEFVZDAaC+vrhNs4Rj7czXSlqr/TCBc5V72nTgd8nP/e4R+9774xHr+StznFFLlUvWkHazqT+/pvBjYq2FwB8tUF/FcAPb/A6DmsEzZj//w3g1wC2EtE0EX0NwLcAPEZEJwB8uvHa4S5GM1bbU1dgfcrnZ3G4g9FSz3YkEsXghrrpajdDLxTY5JzL6MeKdLLZXa6w7kCW8p5fYTO5bPT1Za1WJch0wqrp6u/hcmizqHWHkoisU42vH4/H1bqAcF/Y3WSrIgkuELY886L+emWV9SI7gS8qvrvMvK7tiye4ZPvjD97r0cdP6nbZh47M8r0yOtovW+80Cxdrc/AFbiM5+ILWdiMhwDSaj8sELADIZfkoj1qiIpthU7tUYK90LqPNYtGsDMk2HYzt6+Ijv6ObTfC+Tn2vaohzuPNR/YyLG9n8L1ZnmFG2u6HJJqB61klV1N6RJdo6u9mNUKvyNavWd5VK8TNHLBdeOitEc5lF/f3bdOl4Z5K/nxdf/Jnizc9ZJeJNwJ1IDr7AbSQHX+A2koMvaH3D9ob+ELLGT6WExTmS0nrFhzaxy749xvpBkPTvYDXD+kEhpycxxdu4vn3rBOtLIxv14JpAmIfa2F3fRgYH+RqnOaTT0a3N5e4udimEQro1juybbqwsh1gbd1GrFFgvCljhpLAw/wvQNYA9vZzMtiJq+FfTs2rdcB+HVr7we59RvB/86H9xvXAnkoMvcBvJwRe0VLQl2xJ49MGPAAA2bb9P8S5e4Lzn4SE9y23LxLhHr+vjOrSgNbI0K0zfomWSkxhT2t7G5n97uxZLQdE4PmyJ3/wqR8w/vINF4OiWUbWuLGbKGeu3WqmJcaNB/fxBkWBWLrA8q1nmfyDE16SYvgYET85ECQV1FKBa4u+qr1dPl3zkd7gZ/fPf24tm4E4kB1/gNpKDL2ipaEsk4vjIvfWhwvfs0qItv4PFV1vK6rAhaEN8lAes47q7jb23VsxW/WJqIghascQGynK8lQ7ajm/mYcxxkaedX9UWopEJcaS/YiM80TVrhklVfDY5FqtkT+OuieS4kDUlXHzSrBiifPa0bnz68CPc+SRX1hGChC0um4A7kRx8gdtIDr7AbSQHX9DirrYBxBumd7s15rMtIR4lpF2+0htMUkcistaJTmblmsXji8ikugr0OuElgLE85+1ixHxF1LxVa5aLWkT8DXQRQkDeoGplBoim6kbWw1kj30n0xola9w5X+ZnbREm7mdN61vwpTohbv1V79y8HdB1dM3AnkoMvcBvJwRe0uBlpEMlUXTwYy3TPFfn4NkUdiCwK3uqKnLhdstax6W6P1ywLs74s3mc3J8+JXOmKlSud7Oakt2SKA8mdST2LJBbhQG3V8o6DRDAW2vWQTLKXfeGSmJ2S16KmVuMEOIIVFK7yd9chktc2btDF0HnRlc3UrMS5pC5BbwbuRHLwBW4jOfgCt5EcfEFLdaR0OoMfvPATAEA1/KriLS2xObqybI3NFJaw1Jfm5nRNV1X4Cbr7dLfarl7uWBYVNferizp5bfIEt3HJrGjdZGSMI/5BUVPXkdTd0MbGOJSy3prXNraJR7J3R7X5n4zxNWsyTBTUJn5ZzFoPhvRZEBTXHBhl3S1mDdApG3YhBK3p7N3dOkTVDJop2R4holeI6AgRHSaiP2/83XVtc/DQjGirAPgrY8x2AA8A+DoRbYfr2uYg0Ezt/wyAmQadJaKjAIZxA13bMtkV7H3lNQBA53rdjNRUWYzsf+0Vxdu4nj2vvT0sRi5M6zzkivD4JqzWOCVRTzY3zZHwT+15UK27/957PDpX1PNM5Ny40+e4BHryxEm17uCh/R7dmdJJY1/80u979MP3bFG8iEhZWD/I3dtKlmiTSXp2BkFZeNIDYixptFMn8MWFd78W1C6KG+lidV3KdqMF4C4Ab8B1bXMQaHojEVE7gO8B+AtjjOrJa4wx+K1mid77niaifUS0r1Qqvt8ShzWApjYSEYVR30T/ZYz5fuPPc41ubbha1zZjzDPGmN3GmN2RyPVPb3a4M3BNHYnq4fZ/A3DUGPNtwfpN17ZvocmubV3dPfiDp/4YABDtn1C8XJb1nRMH31O8wXWsLwSEbI9bXWFLNY5wb9mhr981yO6AXC8bmJ//7KfVukSSk/9XLR1JlvFXRKZBoaLXXbrEvQrOnr6oeIkEP/Ps9ILinTl8wqMDosfhqVn9G93zmd0evXFUd6uVroFATNj1YZ2FQDIsYnUAjpAODTWDZvxIDwP4IwAHieg3rem/ifoGer7Rwe0sgC9f990d1gyasdp+BeBKSbyua5sDgBZ7tomAaKQumiaP6ZHsmWUWbcY2acVcsxUR/ScrsS0mxo2WczqhfXmerzl3js3/n/z0J2rdkmivs7yik/qTortbSrTJabO8xtPTLM76e4cVL9bBIvbVH+l7L5444NFVMUJ9alZ78KdFhsLENi3CUx1c9p3q4myFeEKb/6k2/q7C1sjVROL6dVkXa3PwBW4jOfiCloq2WqWM7EJdhL38wx8p3vnZaY8OlHV+8YEDwm0lxFmlYtWkCWtj74svK1YkzMf1/bs+7NGliG76nhEjpk6d09bSwgIHdEsFvtfF2TNq3ekzvG73ro8o3p99/S89+k0xehQAKstsxWVEcl/ectGd2sei+dW3ZxSvLcQiMSwaxwetKdtJIdrWbxxVvCe/+BVcL9yJ5OAL3EZy8AVuIzn4gpbqSOFwBIMD9a5nE6NjimdEfVkooD2rQVXvz3vf1LTuEImJpHWr6fjQEJvhn3icB78kEwm1LhVjr/eRQ9rDPjnFUf51w6MeXbAaDQTjfM1Dk8cU78jkpEcnRrcp3sWLfO+uTqb7IzrzLNHO3vfFWd2IfeHClEfPX2a3QaFquVSEm34mrbfBQ59ytf8OtwhuIzn4gpaKtkqlgsX5ekDzgY89pHgPPfqoR0ej2tMaEuJMBm1liTYABMHvK5d0IDJfYrN+Yfq0Ry8Wymrd4mUOuJ6a0glrFy+x971djvaMajFKERZtpYpOndn7i1959MbxnYo30s3iNyZa4yTC2nQvFtizfSpzWPHak+x9r4o5KLNLOv+8t3fUo3NWefvLv3gT1wt3Ijn4AreRHHyB20gOvqDFbW0IbY3I8kJGJ4PtP/C2R/f368qmgX4xr03U8C8t6Zo0OfA2VNO6z/AY6zQjYhbuhUkdYlhdYZ2mf0DXpCV6uKAgKJLqcnn9WQYHua5t9uK04l1e4IyCwSE9J41E1sOK6GOAkFWTJtvaxHWdflS4SkoL3IUXAZ3SPyDcF6WiTv4375s0fXW4E8nBF7iN5OALWivaCIiG66ZmsaDF0muvveTRpqxFRUeCPblyzlvB6vYaEr+LjaMjirfjge0ePb6BxVz6vBY9s0tcLh6Ja5Ey3sOibn6ezemdW3eodffs5Jq95/7zP6xnZC91eVV/zlKJX5uKcF/EdJaDjOSPjm1SvEvnj/MLMRM1bs2v27aNa+oKOas0fVCXuzcDdyI5+AK3kRx8QWsT22o15PIND7M1Zfvxz36e15W0NRMU4qwmmoAaq5Q5KEZayZFVADCbZjGYTXPgdDGvxQbF2Et9/N1Tirfwa7aCNo2x+ProZp03XRJWXNyq5TPC6rStvYDokiJLn/JW57iQKDnauF6LtsIKJ8dt72CL7s2396t1F8+yCMyv6u/b5JZwvXAnkoMvcBvJwRe4jeTgC1rv2W6v6zEpy3ua7GNztGh1tY2J/R4h1oOMNdY9mmBeraBN2myWCwiComy6f1y3vxlPsPl/4rSO/oNYJwuL2q8LM+fUsh5REi5pACjlWR8pFnXd3KpwBxSFSV4u6s67oRjrfwNDfYp3doaT2ebO8fMXrBq9k4ff9eieHn0N06Xn5TWDZjq2xYjoTSJ6r9Gx7e8afx8jojeIaIqIvktEkWtdy2HtohnRVgTwSWPMfQDuB/AEET0A4B8B/JMxZjOAJQBf++Ae0+F2RzO1/wbAb87ZcOOfAfBJAH/Y+PuzAP4WwL9e7Vq1WgG5bMP0ruk9HCbubDY3p4/hE0fOeHQsxOIsktJiqVcEe4d6U4oXEu6GnhR3fatajTcKeTZ9+/t1txM5InVmlpPcJiePqnWjJc5Ht8V0NsufLZfTpdiZZRa/UrRVS9qDH4yyWX/4kG4WLwOw/f3c+2z4Xu197+9jXm+fDk7Hoh9Qw3YiCjY6kVwCsBfASQBpY7wUvGnU2wE63KVoaiMZY6rGmPsBrAewB8CHmr2B7NiWzeau/QaHOxLXZf4bY9IAXgHwIIBOIm/O5noAF67wHq9jWzKZeL8lDmsAzXRs6wNQNsakiSgO4DHUFe1XAHwJwHNosmMbaga1RoQ7YO3hUJlN646wVlzefv0XHj07x+Y5WUnxe/Zwnf0jD+5WvOVl1k0OvPOGR68WdJhiUrS8OXXmjOLlxQAcI0bFxzq0+ZzJcHJ+dkk3n1/NsA5mV4+FxPj2lPjRDY3pGsCunkGP7h/S+s3QLi4o6BYhkogdTpKvyZo3Zw8EbgLN+JEGATxLREHUT7DnjTEvEtERAM8R0d8D2I96e0CHuxTNWG0HUG+JbP/9FOr6koMDyO6O9oHejGge9X6TvQAuX2P53YLb/bvYaIzpu9ailm4k76ZE+4wxu6+9cu1jrXwXLmjr4AvcRnLwBbdqIz1zi+57O2JNfBe3REdyWHtwos3BF7R0IxHRE0R0vJHDdNcNClzL0zhbJtoanvFJ1EMs0wDeAvCUMeZISx7gNkBjitSgMeYdIkoCeBvAFwD8CYBFY8y3Gj+wLmPMVYco3m5o5Ym0B8CUMeaUMaaEeozuyRbe/5bDGDNjjHmnQWcByGmczzaWPYv65rqj0MqNNAzgvHh9V+cwrbVpnE7ZvgW40WmctzNauZEuAJCdHa6Yw7SWcTPTOG9ntHIjvQVgolF9EgHwFdSnUN41aGIaJ9BsbtdthlZH/z8H4J8BBAF8xxjzDy27+W0AInoEwKsADgJeh/pvoq4nPQ9gAxrTOI0xi+97kdsUzrPt4Aucsu3gC9xGcvAFbiM5+AK3kRx8gdtIDr7AbSQHX+A2koMvcBvJwRf8P3JJ9i1vy1JDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 144x144 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "image, label = imgs_batch[1], labels_batch[1]\n",
    "print(\"The label in the picture is {}\".format(label))\n",
    "plt.figure(figsize=(2, 2))\n",
    "plt.imshow(image.transpose(1,2,0))\n",
    "plt.savefig('cnn-car.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 5.5.1.3 构造Dataset类\n",
    "\n",
    "构造一个CIFAR10Dataset类，其将继承自`paddle.io.DataSet`类，可以逐个数据进行处理。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import paddle\n",
    "import paddle.io as io\n",
    "from paddle.vision.transforms import Normalize\n",
    "\n",
    "class CIFAR10Dataset(io.Dataset):\n",
    "    def __init__(self, folder_path='/home/aistudio/cifar-10-batches-py', mode='train'):\n",
    "        if mode == 'train':\n",
    "            #加载batch1-batch4作为训练集\n",
    "            self.imgs, self.labels = load_cifar10_batch(folder_path=folder_path, batch_id=1, mode='train')\n",
    "            for i in range(2, 5):\n",
    "                imgs_batch, labels_batch = load_cifar10_batch(folder_path=folder_path, batch_id=i, mode='train')\n",
    "                self.imgs, self.labels = np.concatenate([self.imgs, imgs_batch]), np.concatenate([self.labels, labels_batch])\n",
    "        elif mode == 'dev':\n",
    "            #加载batch5作为验证集\n",
    "            self.imgs, self.labels = load_cifar10_batch(folder_path=folder_path, batch_id=5, mode='dev')\n",
    "        elif mode == 'test':\n",
    "            #加载测试集\n",
    "            self.imgs, self.labels = load_cifar10_batch(folder_path=folder_path, mode='test')\n",
    "        self.transform = Normalize(mean=[0.4914, 0.4822, 0.4465], std=[0.2023, 0.1994, 0.2010], data_format='CHW')\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        img, label = self.imgs[idx], self.labels[idx]\n",
    "        img = self.transform(img)\n",
    "        return img, label\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.imgs)\n",
    "\n",
    "paddle.seed(100)\n",
    "train_dataset = CIFAR10Dataset(folder_path='/home/aistudio/datasets/cifar-10-batches-py', mode='train')\n",
    "dev_dataset = CIFAR10Dataset(folder_path='/home/aistudio/datasets/cifar-10-batches-py', mode='dev')\n",
    "test_dataset = CIFAR10Dataset(folder_path='/home/aistudio/datasets/cifar-10-batches-py', mode='test')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.5.2 模型构建\n",
    "\n",
    "使用飞桨高层API中的Resnet18进行图像分类实验。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0503 13:12:10.322129   156 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1\n",
      "W0503 13:12:10.327652   156 device_context.cc:465] device: 0, cuDNN Version: 7.6.\n"
     ]
    }
   ],
   "source": [
    "from paddle.vision.models import resnet18\n",
    "\n",
    "resnet18_model = resnet18()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "******\n",
    "\n",
    "飞桨高层 API是对飞桨API的进一步封装与升级，提供了更加简洁易用的API，进一步提升了飞桨的易学易用性。其中，飞桨高层API封装了以下模块：\n",
    "1. Model类，支持仅用几行代码完成模型的训练；\n",
    "1. 图像预处理模块，包含数十种数据处理函数，基本涵盖了常用的数据处理、数据增强方法；\n",
    "1. 计算机视觉领域和自然语言处理领域的常用模型，包括但不限于mobilenet、resnet、yolov3、cyclegan、bert、transformer、seq2seq等等，同时发布了对应模型的预训练模型，可以直接使用这些模型或者在此基础上完成二次开发。\n",
    "  \n",
    "飞桨高层 API主要包含在`paddle.vision`和`paddle.text`目录中。\n",
    "\n",
    "******"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.5.3 模型训练\n",
    "\n",
    "复用RunnerV3类，实例化RunnerV3类，并传入训练配置。\n",
    "使用训练集和验证集进行模型训练，共训练30个epoch。\n",
    "在实验中，保存准确率最高的模型作为最佳模型。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/nn/layer/norm.py:653: UserWarning: When training, we now always track global mean and variance.\n",
      "  \"When training, we now always track global mean and variance.\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Train] epoch: 0/30, step: 0/18750, loss: 7.28671\n",
      "[Evaluate]  dev_score: 0.05540, dev_loss: 12.52092\n",
      "best accuracy performence has been updated: 0.00000 --> 0.05540\n",
      "[Train] epoch: 4/30, step: 3000/18750, loss: 0.77707\n",
      "[Evaluate]  dev_score: 0.65140, dev_loss: 1.01241\n",
      "best accuracy performence has been updated: 0.05540 --> 0.65140\n",
      "[Train] epoch: 9/30, step: 6000/18750, loss: 0.56135\n",
      "[Evaluate]  dev_score: 0.72050, dev_loss: 0.83436\n",
      "best accuracy performence has been updated: 0.65140 --> 0.72050\n",
      "[Train] epoch: 14/30, step: 9000/18750, loss: 0.54198\n",
      "[Evaluate]  dev_score: 0.70990, dev_loss: 0.88424\n",
      "[Train] epoch: 19/30, step: 12000/18750, loss: 0.75522\n",
      "[Evaluate]  dev_score: 0.71290, dev_loss: 0.89758\n",
      "[Train] epoch: 24/30, step: 15000/18750, loss: 0.61934\n",
      "[Evaluate]  dev_score: 0.72110, dev_loss: 0.83567\n",
      "best accuracy performence has been updated: 0.72050 --> 0.72110\n",
      "[Train] epoch: 28/30, step: 18000/18750, loss: 0.69278\n",
      "[Evaluate]  dev_score: 0.72720, dev_loss: 0.82229\n",
      "best accuracy performence has been updated: 0.72110 --> 0.72720\n",
      "[Train] Training done!\n"
     ]
    }
   ],
   "source": [
    "import paddle.nn.functional as F\n",
    "import paddle.optimizer as opt\n",
    "from nndl import RunnerV3, Accuracy\n",
    "\n",
    "#指定运行设备\n",
    "use_gpu = True if paddle.get_device().startswith(\"gpu\") else False\n",
    "if use_gpu:\n",
    "    paddle.set_device('gpu:0')\n",
    "#学习率大小\n",
    "lr = 0.001  \n",
    "#批次大小\n",
    "batch_size = 64 \n",
    "#加载数据\n",
    "train_loader = io.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\n",
    "dev_loader = io.DataLoader(dev_dataset, batch_size=batch_size)\n",
    "test_loader = io.DataLoader(test_dataset, batch_size=batch_size) \n",
    "#定义网络\n",
    "model = resnet18_model\n",
    "#定义优化器，这里使用Adam优化器以及l2正则化策略，相关内容在7.3.3.2和7.6.2中会进行详细介绍\n",
    "optimizer = opt.Adam(learning_rate=lr, parameters=model.parameters(), weight_decay=0.005)\n",
    "#定义损失函数\n",
    "loss_fn = F.cross_entropy\n",
    "#定义评价指标\n",
    "metric = Accuracy(is_logist=True)\n",
    "#实例化RunnerV3\n",
    "runner = RunnerV3(model, optimizer, loss_fn, metric)\n",
    "#启动训练\n",
    "log_steps = 3000\n",
    "eval_steps = 3000\n",
    "runner.train(train_loader, dev_loader, num_epochs=30, log_steps=log_steps, \n",
    "                eval_steps=eval_steps, save_path=\"best_model.pdparams\")\n",
    "\n",
    "                "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "可视化观察训练集与验证集的准确率及损失变化情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmUAAAFECAYAAABml0kpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XecVOXZ//HPNWUr7FJ2l7KgIFIWQdEg+lgCGomaR03UaDAmEWNPLFGfxJJo/GmssbcYW4yJSuwlsWvEXrAuuCigIm0LdZetU+7fHzMsCyy6CzNzZme+79drX8ycc/Y+X1CGa+9zn+uYcw4RERER8ZbP6wAiIiIioqJMREREJC2oKBMRERFJAyrKRERERNKAijIRERGRNKCiTERERCQNqCgTERERSQMqykRERETSgIoyERERkTQQ8DrAligpKXHDhg3zOoaIpND777+/3DlX6nWOraXPL5Hs09XPrx5ZlA0bNoxZs2Z5HUNEUsjMFnqdIRH0+SWSfbr6+aXLlyIiIiJpQEWZiIiISBpQUSYiIiKSBlSUiYiIiKQBFWUiIiIiaaBH3n0p2am+vp7a2lpCoZDXUSTBCgsLGTJkCD6ffk4Ukeylokx6hPr6empqaigvLyc/Px8z8zqSJEg0GmXJkiUsX76csrIyr+OIiHhGP5ZKj1BbW0t5eTkFBQUqyDKMz+djwIABrFmzxusoIiKeUlEmPUIoFCI/P9/rGJIkwWCQcDjsdQwREU+pKJMeQzNkmUv/bUUk3bholGh9C+FFq4lUN8S2OUfjAx8T+qwuKefM6DVl4a9Wsea85+h15p7kThrqdRwRERFJAReN4upbia5pIbqmBfP7CO4wAIDGez8gsqS+fZ9b00Jwx0EUnTsZgOqdbiS8cDWuvhWcA6DgFzvT/+9HYGasPPYR+lx1AMHRiX8Ub0YXZZbjp2nGJ+Tsta2KMhERkR4i2tRGdHkT0VXNRFc3Q9SRt88IABr/8SGhymqi9a24eGHlH9SbfncdDkDtd2+n9bWvNhgvZ69tGfDaSQDUXz6T8Nw6CPrxFefiK87D13f98pjc/bYnNxTBV5yHFefhK84jOHb9TUgDPzkd/8BeSfl9Z3RR5hvUGyvKJVyVnGlGEa/NnTuXiooK3nvvPSZOnLjF4zz77LMceOCB1NXVUVJSksCEIqnT9skycnYcBEDDda/T9u5ifIOL8A/ujX9wEYEhReTuPdzjlNnHtYWJrmzGP7A3AK1vf03ow6VEljcRXd5IdHkTri1CyUM/BWD5tAdo/lflBmP4y4sYvPhcAJpmfELLSwvaCyorzsNXtr5Iyv/JeHL33Q5f0bqiKhf/kOL2/WVvnITlB7G8QKdLJ/pe84Nv/P0ERyXvMzKjizIzIzimlFBVrddRJEt921qpbbfdlq+++mqLxx85ciTLli1TISVZLbKyidW/+Q9N//iQ0pePI2+fEUTqGmmbtYTIkipcc6y3Ycd/2FccNYO2D5fiLy/CPzj2FawopXD6d2Jj1jTE/sHPC3r2+0pX0YZWIkvWEF3etL6wqmuk12/2xJcfZO1ds2i8/d32fa6+FYAhrRdjOQGa7v+YtTe9BYAV5+EvKcBX1gvnHGZGwY/HkTNhML7SQnx9YzNVvn4F7ecveeJnWMC/2Xy9f/0/35jf32GsdJPRRRlAoKKM1hfmex1DstSyZcvaX7/55pscfvjhfPDBBwwaFPtp3u/v/IOlra2NnJycbx3f7/czcODAxIQV6YGan/iUlSc/QbSukaI/7EPuHtsC0Oey/elz2f4453D1rUSW1hNtaG3/vuDEclwkSmRJPa2vLySytJ6cnQe3F2V1B9xD6KNl+Prm44vPtOVNHk7R7/cBoOXF+VivnFhBN7AXltNz/jl1kSiuKYRrDeNawrjmENGVzQQrSvEV5dH20VKa/lXZPosVif9a+u9fEBjRn8Y732P1WU9vMm7BUTvhG9YXC/jw9cknsH1/fCUF+EoL8ZcUQmx5FkUX7EPR+VPw9cvv9M+t4MfjvzH/NxVkPV3P+b9oC+XsPIjQx8twrWEsN+N/u5JmOhZM/fr1A6C0tHSTQmrgwIGccsopLFmyhIcffpjx48czc+ZMrr76au69914WLFhAcXEx++67L9dee217k9WNL1+ue//II49wxx13MHPmTAYPHszFF1/MT3/6025lf/311zn33HOZNWsWBQUFHHTQQVx33XX0798fgIULF3LGGWfw2muv0dTURHl5OaeddhpnnHEGAA8//DCXXHIJ8+bNIzc3lzFjxnDHHXcwbty4Lf7zFOlo5SmP03jbuwR3GkTp08eQs/PgTY4xs/Z1QR0Vnb33Bu+dc7jGtvX7z51MeP4KIssaiCytJ7K0gfCS+vb9K46aQXR5U/t7X2khhcfsQp8/HwjAmj+9jL9/Af7BResvoZYWYjmBWDG4cDWuJYRrjbQXR4FhfQkM60u0oZXmxz+NbY/vozVC3ve3J2fiEMKLVlN/6Su41jCsK6xaI/Q+ey/y9h1B2/tLWHnsI+vHj//a/74jyf/fMbQ88znLD753kz+rdbOM4c+X03DN6/hKCvCXFOIrKSA4fgDEZ/7zDhxFv4G9YzNc8f2+/gX4CmI/SBYeswuFx+yy2f9u/tLkrMfKBBlfpfQ+Y096n7Gn1zEkCV75zR3UffRlys9bOmE4U64/IeHjXnPNNZx77rm88847RCIRIPYPyvXXX8/w4cNZunQpZ555Jj//+c957rnnvnGsc845hyuvvJKbbrqJ2267jenTp7PHHnswbNiwLmVZtGgR+++/P0cccQR//etfWb58OSeffDLTpk3jhRdeAOCEE07A7/fz8ssvU1xczIIFC1ixYgUAX3/9NdOmTeOaa67hkEMOobm5mffff3+zM4Mi3bHuMlfOpKH4B/am6LzJWz1TZWZYr9z29wU/2fEbjy994ZdElq4r2GJFW2B0bBmBawlR/8eXIOo2+J7e50+hz6XfJ7qyiWUjrt5kzOJLv0/R+VOILm9k5S8e2jRj7xxyJg7BNbTS/NgcLDcQWxeVG4C8QPtlWisMEhjRD8sLQPsxfvxDY+uqgjuUUXz1gbHvj+/39csnGF+Pl3/4OIYcMX6zyy+CY8oIjtHTN5Ih44sykZ5i77335vzzz99g29lnn93+evjw4dxwww3ssccerFixon3GqjNnnnkmhx12GACXXXYZN910EzNnzuxyUXbjjTcyYMAA7rzzTgKB2MfEPffcw+677867777LpEmTWLhwIcceeyw77bQTwAZjL1myhGg0ypFHHtl+qXbs2LFdOnc6MbMDgBsAP3Cnc+6KjfZfB+wTf1sAlDnn+qQ2ZfaI1K1l1alPkbff9vQ6YVd6Hfsdz7LkTBgMEzrfZ3lBhrReTLS2sb1giyytJzAm1kLBV5xHv3t+vL5oyvVjeQECI2J/p/1Dihk47+zY9o6FV07sh5rg2AGU1/x+s9mCY8ooeexnm90fGN5vk5nCDfL71cLUKykryszsbuAgoNY5Ny6+7c/AwUAbsAA41jm3OpHndc5RN/Vu8vYdQdH5UxI5tHgsGbNVXpo0adIm21588UWuvPJK5s6dy+rVq4lGo0Ds0uE3FWUTJqz/1yInJ4eSkhJqamq6nGXOnDnsscce7QXZunx5eXnMmTOHSZMmcdZZZ3HqqafyxBNPMGXKFA466CD23DM2K73rrrsyefJkRo8ezdSpU5kyZQqHHXYY5eXlXc7gNTPzA7cAU4HFwHtm9qRz7tN1xzjnzuxw/GnAzikPmgWcczQ/VMmqXz9JdE0rubunf4sjC/jbbyDYZF9O4Bsv71nQT3D7zf/9lsyVynL4HuCAjba9AIxzzu0IfA6cl+iTmhmR6gZa31mU6KFFEqqwsHCD9/Pnz+eggw5i9OjR/Otf/2LWrFk89FDskkZbW1tnQ7Tb+CYBM2sv6BLlpJNO4ssvv+S4447j66+/ZurUqRx//PEABAIBXn75ZZ5//nl23nlnZsyYwciRI9svffYQk4D5zrkvnHNtwAzgh99w/FHAAylJlkUiNQ2s+PH9rPjJDPzD+jLgg1/T+8y9vI4lkhQpK8qcc68CKzfa9rxzbt0D794GhiTj3MGKMsJqiyE9zDvvvEMoFOL6669njz32YPTo0VRXV6fk3DvssANvvvnmBs+jfPfdd2lpadlgof6QIUM4/vjjue+++7j11lu5++67aW2N3eFmZuy+++784Q9/4I033mDSpEncc889KcmfIOVAx5/mFse3bcLMtgWGAy9vZv+JZjbLzGbV1alvYne0fbSM5v98RvEV+zPgrZPJGae7jSVzpdOF418Cz2xu59Z8qAXHlBJesDJ2p4pIDzFq1Cii0SjXXXcdX375JY888giXX355Ss59xhlnUFNTw/HHH8+cOXOYOXMmxx57LPvttx+77rorACeffDLPPvssCxYsYPbs2Tz++OOMGDGC3NxcXnnlFS677DLeffddvv76a55//nk+/fTTHrmurIumAQ875yKd7XTO3e6cm+icm1hamvhHs2SayLJ6mv71CQD5+49i8Fe/peicyRndCkEE0qQoM7PfA2Hgvs0dszUfaoGKUog6wvNXbGVSkdTZddddufbaa7nhhhsYO3YsN910E9ddd11Kzj1kyBCee+455s2bx3e+8x0OPfRQJk6cyIwZM9qPiUQinHbaaYwbN47JkycTiUR46qmnAOjbty+vvvoqBx98MCNHjuTEE0/kuOOO45xzzklJ/gRZAnRcvDQkvq0z09Cly63mnKPx7x+wbOz1rDzhMaKrmgHaO8GLZDpzzn37UYk6mdkw4N/rFvrHt00HTgK+55xr6vw7NzRx4kQ3a9asLp839GkNq3/3LMWX70/OeE1990RVVVVUVFR4HUOS6Nv+G5vZ+865LX+WVDeZWYDYWtfvESvG3gN+6pybs9FxY4BngeGuCx+o3f38yhbhxWtYddLjtDz9GTl7bUu/uw5P6uNsRFKpq59fnrbEiN9u/jtgclcLsi0RHDuA0n8fk6zhRSQDOefCZnYq8Byxlhh3O+fmmNnFwCzn3JPxQ6cBM7pSkEnnog2t1Ox8E64xRJ8bDqLXqbtjvrS4kCOSUqlsifEAMAUoMbPFwB+J3W2ZC7wQb1L3tnPu5GRlcJGo+q+ISJc5554Gnt5o24Ubvb8olZkySWRlE/5+Bfh659Lnmh+Qu+e27b26RLJRyooy59xRnWy+K1XnX3HMQ4Tm1DJw1q9TdUoREemEc47G299l9W+fpf+/ppF/4GgKf7H5vl0i2SJrOvr7+hUQrqrFRaOaFhcR8Uj4q1WsPP5RWl9aQO6+2xEco7tRRdbJmqIsWFGKawoRWbSGwLZ9vY4jIpJ11t49i9Wn/xt8Rt+//ojCE3bd7PMVRbJR1hRl6545FqqqU1EmIuKFqCN3r23pe/uhBLbRI0JFNpY11/GCFbGiLDxX3bRFRFLBRaM03PAGjfe8D0DhcRMpeWa6CjKRzciaosxf2otev9mT4PgBXkcREcl4oc+XU/vdO1j9m//Q8tw8IPboLV2uFNm8rLl8CdD3uv/1OoKISEZzkSgN179B/R9egLwA/f7+Ywp+vrPXsUR6hKyZKYPYh0V4gR61JNln+vTp7Lfffl7HkCzQ9tbXrPm/Z8j9/kgGffobCn+xi2bHRLooq4qytTe+ybLtryGyvNHrKJJFpk+f3n7ZJhgMUlJSwl577cVVV11FY6P+X5Sez4UjtL7+FQC5ew2j7K2TKXn8Z/gHFXkbTKSHyaqibN0dmFrsL6m29957s2zZMhYuXMh///tfjj76aG6++WZ22WUXampqvI4nssXaZldTu8dfqZ1yZ/uViNzdt9HsmMgWyKqiLFhRBsTaYoikUk5ODgMHDmTw4MGMHz+eU045hbfeeou6ujrOPffcDY696aabGDNmDHl5eYwcOZJLL72UcDgMwO9//3tGjx69yfinnHIKe+21V5fzOOe4+uqr2W677cjJyWHEiBFcf/31GxzzxBNPsPPOO1NQUECfPn2YNGkSH374IQChUIizzjqLIUOGkJuby6BBg5g2bVp3/1ikB3OhCPWX/peaXW4h/OVK+t9/JP7t+nkdS6RHy6qF/v5tirH8IKGqWq+jSILUTrljk235R46n9692J9rUxvIf/H2T/YXTd6Fw+neILG9kxY/v32R/r1N2o+AnOxJetJqVP39ok/1lr5yQkOzl5eUcffTR3Hvvvdx11134fD4uuugi/va3v3H99dczYcIEqqqqOPnkk2lpaeGSSy7hmGOO4bLLLuOdd95ht912A6C1tZV//etfXHHFFV0+96233soFF1zADTfcwD777MNLL73Eb37zG3r37s1xxx1HdXU1RxxxBH/605844ogjaGlp4cMPPyQQiH1k3HTTTTz44IP885//ZLvttqOmpoY33ngjIX8ukv5cJErt3rfT9s4i8o8cT9+bD8Zf2svrWCI9XlYVZebzERhdosuXkjZ22GEH6uvrWb58Ob169eKqq67i0Ucf5YADDgBg+PDh/OlPf+L000/nkksuYdSoUey2227ce++97UXZU089RXNzM0ceeWSXz3vFFVdw2mmnceKJJwIwcuRIPvvsMy699FKOO+44li1bRigU4sgjj2TYsGEAVFRUtH//woULGTVqFJMnT8bM2Gabbdh1110T9Kci6c78Pgqm7Ujv3+1NwWHjvI4jkjGyqigDKLpwX6wg6HUMSZBvmrXyFeR8435/SeE37g8M7ZOwWbHNcc4Bsf5Nc+bMobm5mcMPP3yD9TiRSISWlhbq6uooLS3lmGOO4YILLuD6668nGAxy7733csghh9CnT9cactbX17N48WK++93vbrB98uTJ3HDDDTQ1NbHjjjuy//77M27cOKZOncqUKVM47LDDGDp0KADHHnssU6dOZfvtt2fq1KlMnTqVgw8+mJycnAT9yUi66/2bPb2OIJJxsmpNGUDBoTuQv/8or2OIADBnzhyKi4vp378/0WgUgIceeoiPPvqo/auyspJ58+bRr19svc60adNoaGjgP//5D3V1dTz77LMcc8wxCc3l9/t55plnePnll9l111155JFHGDVqFP/+978BmDBhAl9++SVXX301OTk5nHHGGUyYMIH6+vqE5pD0VH/FTFrf/trrGCIZJ+uKsmhTGy2vfEGkdq3XUSTLLVmyhPvuu4/DDjsMn8/HDjvsQF5eHl988QXbb7/9Jl9+vx+Avn37cvDBB/OPf/yDBx54gH79+rH//vt3+bxFRUUMGTKEV199dYPtM2fOZPjw4RQUFACx2btJkyZx/vnn8+qrrzJ58mT+9re/tR/fq1cvDj30UG688UZmzZpFVVUVM2fOTMCfjKSz6Kpm1pz3HK2vfuV1FJGMk3WXL8MLVlK3z530u/8nFB61k9dxJEu0tbVRXV1NNBplxYoVvP7661x++eWUlZVx+eWXA7Ei5/zzz+f888/HzNhvv/0Ih8NUVlby4YcfcuWVV7aP94tf/IIjjjiCqqoqjj766PaCravOO+88zj77bEaOHMmUKVN4+eWX+ctf/sItt9wCwJtvvslLL73E97//fQYNGsS8efP45JNPOO644wD485//zODBg5kwYQIFBQU88MAD+P1+Ro3SLHSmC82uBtAj60SSIOuKsuCoEvAZYd2BKSn02muvMWjQIPx+P8XFxVRUVHDqqafy61//msLCwvbjLrjgAgYNGsTNN9/M2WefTX5+PqNGjWL69OkbjHfggQdSXFxMVVUVDzzwQLfznHLKKTQ2NnLZZZfxq1/9iqFDh3LFFVe0F13FxcW89dZb3HLLLaxatYqBAwdy9NFHc8EFFwCx2bZrr72WefPmEY1Gqaio4JFHHum0XYdklrbKWF+94PiBHicRyTy2bqFxTzJx4kQ3a9asLf7+ZSOvIThhECUP/TSBqSSZqqqqNrj7TzLPt/03NrP3nXMTUxgpKbb288trK095nKYZn1C+8gI1iBXpoq5+fmXdmjKIdfZXrzIRke4Lz19JzviBKshEkiDrLl8CBCtKaXl+Hi4cwQLdW4sjIpLNSp8/FremxesYIhkpK4uywhMnkX/4ONBPeiIi3WJmWJ98r2OIZKSsvHwZ3L4/ubsNxfxZ+dsXEdkirW8sZOUvHyGyTP3oRJIhK6sS5xyN931E62tfeh1FuqEn3pQiXaP/tj1D62tf0fi397F8PRVFJBmysigzM1b/3zOsvft9r6NIFwWDQZqbm72OIUkSCoXaH3Yu6StUWY1/aDE+Xb4USYqsLMogtthfDybvOcrKyliyZAlNTU2aVckw0WiUmpoaiouLvY4i3yI0u0b9yUSSKGt/NA1WlNJ438c453Rrdw9QVFQEwNKlSwmFQh6nkUQrLCykpKTE6xjyDVwoQqiqjrwD9dQGkWTJ2qIsUFGGW9NCtLoB/6Air+NIFxQVFbUXZyKSWpHatQS260vOhEFeRxHJWNl7+XJMKQChKl3CFJHOmdkBZvaZmc03s3M3c8yRZvapmc0xs/tTnTFVAuXFDJp7FgXT9MxgkWTJ2pmynD23ZdDic/AP1syLiGzKzPzALcBUYDHwnpk96Zz7tMMxI4HzgD2dc6vMrMybtCKSCbJ2psyXHyRQXqz1ZCKyOZOA+c65L5xzbcAM4IcbHXMCcItzbhWAcy5jn9+28vhHWXny417HEMloWVuUATT+80Pqr3rV6xgikp7KgUUd3i+Ob+toFDDKzN4ws7fN7IDOBjKzE81slpnNqqvrmUsmWl5aQHS12tKIJFPKijIzu9vMas1sdodt/czsBTObF/+1b6ryALS8MJ+1N7yZylOKSGYJACOBKcBRwB1m1mfjg5xztzvnJjrnJpaWlqY44taLNrQS+WoVOWqHIZJUqZwpuwfY+KfIc4GXnHMjgZfi71MmOKaUyNJ6onq4rohsagkwtMP7IfFtHS0GnnTOhZxzXwKfEyvSMkpodg2AepSJJFnKijLn3KvAyo02/xD4e/z134EfpSoPxNpiAIQ+65mXE0Qkqd4DRprZcDPLAaYBT250zOPEZskwsxJilzO/SGXIVAhVVgMQHDfA4yQimc3rNWUDnHPL4q+rgc3+jU/GmoxgRewyQlhtMURkI865MHAq8BxQBTzonJtjZheb2SHxw54DVpjZp8B/gd8651Z4kzh5fP0LyDtwFP5hm1yZFZEESpuWGM45Z2abfX6Oc+524HaAiRMnJuQ5O4Ht+mEFQSK1axMxnIhkGOfc08DTG227sMNrB5wV/8pYBYePo+DwcV7HEMl4XhdlNWY2yDm3zMwGASm9ndyCfsrXXIgF/Kk8rYhIj+Gcg1AEy/H6nwuRzOf15csngWPir48Bnkh1ABVkIiKbF61uYHHhRTT+40Ovo4hkvFS2xHgAeAsYbWaLzew44ApgqpnNA/aLv0+p5n/Ppfb7d+Pawqk+tYhI2gtV1kA4in+Inn4ikmwpm492zh21mV3fS1WGzkTXtND6wnzC81cQHKs7i0REOmpbd+el2mGIJJ3Xly89pweTi4hsXqiyBt/A3vhLCr2OIpLxsr4oC8SLsvBcFWUiIhsLza4hOF5XEURSIetvp/EV5uDfpo9mykREOlH4i53xlRR4HUMkK2R9UQaQO3k4vqJcr2OIiKSd3qfv4XUEkayhogzof+8RXkcQEUk7kZoGXDiKf3ARZuZ1HJGMl/VrykREpHNr//IOy4ZehWsOeR1FJCuoKANCn9awbMy1tLw43+soIiJpI1RZQ2D7fvgKcryOIpIVVJQBvtJCwp8tJzS7xusoIiJpI1RZrf5kIimkogzwlRTi65dPqCqlj94UEUlb0aY2wvNXqh2GSAqpKAPMjEBFmXqViYjEhT+tBec0UyaSQirK4oIVpepVJiIS5x/ej37/PJLcPbf1OopI1lBLjLi8743AhaO4UAQL+r2OIyLiKX//AgqPnuB1DJGsoqIsrmDaThRM28nrGCIiaaH52c/xDykiZ5wuX4qkii5fduCcw7WoH4+IyMpjH6Hh6te9jiGSVVSUxTnnWDrgMtb84QWvo4iIeCqyvJFodYPuvBRJMRVlcWaGf3CRFvuLSNYLVVYDkKM7L0VSSkVZB4ExJepVJiJZL1QZa6StmTKR1FJR1kGwoozIV6uJ6jlvIpLFQpXV+PoX4BvY2+soIllFRVkHwYpScI7w58u9jiIi4pniKw6g9IVfYmZeRxHJKmqJ0UHOrkMo+sM++PrkeR1FRMQz/v4F+PsXeB1DJOtopqyDwPB+FF8ylcC2fb2OIiLiiciyetZc8jLhr1Z5HUUk66go20h0dTOhebp8KSLZqe29JdRf+CKRZQ1eRxHJOirKNrLi5w+x4vD7vY4hIuKJde0wguN056VIqqko20iwopTQ58txkajXUUREUi40uwb/sL74eud6HUUk66go20hgTCm0hgl/qfUUItnOzA4ws8/MbL6ZndvJ/ulmVmdmH8W/jvciZyK1VVarP5mIR1SUbSRYUQZAeK46+4tkMzPzA7cABwJjgaPMbGwnh/7LOTch/nVnSkMmmAtHiCxaQ1Cd/EU8oZYYGwmOKQUgVFVL/kFjPE4jIh6aBMx3zn0BYGYzgB8Cn3qaKoks4Kd85QW4lrDXUUSykmbKNuLrm0/fuw8n/2AVZCJZrhxY1OH94vi2jR1uZp+Y2cNmNrSzgczsRDObZWaz6urSexbe/D58hTlexxDJSirKOtHr2O8QHFPmdQwRSX9PAcOcczsCLwB/7+wg59ztzrmJzrmJpaWlKQ3YHWtvf5dVpz/ldQyRrKWirBPhxWtoenQ2zjmvo4iId5YAHWe+hsS3tXPOrXDOtcbf3gl8J0XZkqL5iSpaX/nS6xgiWUtFWSeaH53DisPvJ1qz1usoIuKd94CRZjbczHKAacCTHQ8ws0Ed3h4CVKUwX8KFdOeliKfSoigzszPNbI6ZzTazB8zM04dPBivWL/YXkezknAsDpwLPESu2HnTOzTGzi83skPhhp8c/uz4GTgeme5N260VXN+vOSxGPeX73pZmVE/swG+ucazazB4n9RHqPV5kC8Tsww3PrYJ8RXsUQEY85554Gnt5o24UdXp8HnJfqXMkQml0DoJkyEQ+lxUxQ/8jbAAAgAElEQVQZseIw38wCQAGw1Msw/iHFWK8cQlXpfZeUiEiiRBta8Q/rq5kyEQ95PlPmnFtiZlcDXwPNwPPOuee9zGRmBMaUqigTkayRf+Bo8r/8rdcxRLKa5zNlZtaXWEPG4cBgoNDMftbJcSnt89PvjkPpd+ehST+PiIiICKRBUQbsB3zpnKtzzoWAR4E9Nj4o1X1+ciYMJrBt36SfR0TEa845qifcRMOtb3sdRSSrpUNR9jWwu5kVmJkB3yMNbiuP1DTQcP0bhL9Y6XUUEZGkiiypJ/TxMq9jiGS9dFhT9o6ZPQx8AISBD4HbvU0F0VXNrD7zP/j6FxDYrp/XcUREkiZUWQ1AcJzuvBTxkudFGYBz7o/AH73O0VFgRH8I+NSrTEQyXqgy1g4jR3deingqHS5fpiUL+gmM7E9Yd2CKSIYLVVbjLy/C1zff6ygiWS0tZsrSVXBMKaFPNVMmIpktuMMA/AN7ex1DJOupKPsGgYoymv/zGS4UwYJ+r+OIiCRF0bmTvY4gIujy5Tcq+u3elK++UAWZiGQs1xbGRaJexxARVJR9I1+ffHz5Qa9jiIgkTfPjn7Kk10WE5mqphojXVJR9AxeNsvq3z9D04CdeRxERSYpQZQ0uFCUwTM2yRbymouwbmM9H04xPaH5qrtdRRESSIlRZTWBkfyxPVwVEvKai7FsEKvRgchHJXG2VNQTHq2msSDrI6KKs9sMFPHHIn1hRtWiLxwhWlBKeW4eLaiGsiGSW6NpWIl+sJKimsSJpIaOLsua6er546l1aVjZs8RjBMaW4xjYii+sTmExEJA2EoxRdsh95+4/0OomIkOl9yiz+q9vyIQIVZfhKC4lUNxDYpk9CYolI6phZMbA9UOmca/M6Tzrx9cmn+A/7eh1DROIyeqYMi1Vlzm15VZY7eTjltb8nd9LQRKUSkRQwswIz+zuwCngXGBLffpuZ/cHTcGkiNH8Fkbq1XscQkbiMLsosXpSxFUVZ+xgi0tNcAuwETAaaO2x/FjjMk0RpZtVJj7H8oHu9jiEicRldlJGgemrNRS+y8oRHEzOYiKTKYcCpzrnX2HARw6fAdt5ESi+hyhot8hdJI5ldlK2zFWvKACKL1tD8ZFVisohIqgwEOrv1Okimr6ftgkhNA9G6RrXDEEkjGV2UWQLWlEFssX+0tpHIyqZExBKR1JgL7NnJ9h8CH6c4S9oJVdYAEBynokwkXWT2T4sJWFMGsV5lAOG5dfj32HZrU4lIalwJ3GJmvYgtZphiZicDpwNHeposDYQqqwF0+VIkjWxVURb/sJsMfO6cm5eYSImTqJmy4JhYURaqqiNXRZlIj+Ccm2FmecAfgQLgTmKXM493zj3pabg0kH9IBb6SQvxlvbyOIiJx3SrKzOx+4G3n3I1mFgTeASqAsJkd5pz7dzJCbrEE9CkD8A/rS85uQ7G8zJ5YFMkUZuYDRgIPOefuMbMSwOecq/U4WtoIjOhPYER/r2OISAfdXVM2BXgj/vpgoDcwCLgIuCBhqRIlQTNl5vcx4O1TKDx6QiJSiUjyOaCS2OcTzrnlKsjWc9Eojf/8kPCi1V5HEZEOuluU9QNq4q+nAo8652qA+4nNmKWVRPQpE5Gex8V+ElsA9N2acczsADP7zMzmm9m533Dc4WbmzGzi1pwvVcJfrGLlzx+i5YX5XkcRkQ66W5TVAcPjr6cC/42/LgDS74ndCbp8CbD2zvdYUnYp0ebQ1g8mIqnwR+AqMyvfkm82Mz9wC3AgMBY4yszGdnJcb+AMYss5eoT2Rf6681IkrXS3KHsIuM/MXgSKgBfi2ycAabvQPxF8vXKI1jUS/nx5wsYUkaS6FJgELDSzJWb2ecevLnz/JGC+c+6L+DMzZxBrp7GxS4jd6dmSsORJFqqsBjOCO6goE0kn3V25/jtgMbANcLZzbl3jrsHAHYkMlkhbu6YMYr3KINYWI2enQVs9nogk3T+38vvL2bD57GJgt44HmNkuwFDn3H/M7Ldbeb6UCVXWENiuL77CHK+jiEgH3SrKnHNh4NpOtl+dsESJlMA1ZYFRJWBGqEprhUV6Aufc/0vm+PE7PK8Fpnfh2BOBEwG22WabZMbqklBltfqTiaSh7rbE2AkIO+fmxN//ADgWmAP8KV60pY1ErvP35QfxD+9LqKpu6wcTkZQxs72AHeJvK51zb3bxW5cAQzu8HxLftk5vYBzwSnypxEDgSTM7xDk3q+NAzrnbgdsBJk6c6PmdR2UvH4/T+liRtNPdy5d/Ba4H5pjZEOBhYCZwApAHbPbuJE8k+O7Lwum74CvKTchYIpJcZlZKbB3sd4E18c3FZjYTOMI5920LRN8DRprZcGLF2DTgp+t2OufWACUdzvcK8H8bF2TpyD+4yOsIItKJ7i70Hw18GH99GPCec+5A4BfATxIZLBES1dF/neIL9qX3GZ09Sk9E0tD1xFpiTHDO9XXO9QV2Jtba57pv++b4zP+pwHNAFfCgc26OmV1sZockMXdStb72JfVXziTa1OZ1FBHZSHdnynJYf4fRFOCZ+OvPiU3dp5f2mbLEDRltbMMCPixX3f1F0tyBwEHOuU/WbXDOfWxmvwa69Jgl59zTwNMbbbtwM8dO2fKoqdP8RBUNN79N77P38jqKiGykuzNlnwE/NrNtiPUpezG+fRCwKpHBEqL96mViqrLW9xazpNdFtLyohosiPUAe0FnL+lVA1q5DCM2uIVhRigX8XkcRkY10tyj7f8BlwJfA6x3WTnyf9Zc100Yi+5QBBEb0A9Bif5Ge4T3gPDNrn9aOvz4vvi8rhSprdOelSJrqbkuMJ+KzZIOATzrsegl4NJHBEipBM2X+fgX4ygoJz1VRJtIDnENsPdgXZvZ2fNvuxBpff9+zVB6KrGwisrSe4Hg1jRVJR92dKcM5V+Oc+wjIMbO8+La3nHOfbmkIM+tjZg+b2VwzqzKz/9nSsTYaeF3ohAwHEKwoU68ykR7AOfc2MAr4O+CPf90DjHLOvethNM+EF6yEoF8zZSJpqtur1c3sWOD3wLD4+y+BS51z92xFjhuAZ51zPzazHGLP0txqyXgeeWBMKc0PVuKcS/jlURFJLOdcDXCB1znSRe6uQxiy9o/rPxxFJK10t3nsGcAVwF+I9SeD2F2Yt5pZb+fcTd0NYGbFxPoITQeIP2MuMfdqJ6EqK/jJeIJjyyASBS2UFUlbZjYdaHLOPbjR9iOBPOfcvZ4E85jl6M5xkXTV3cuXpwFnOOfOcs49Ef86EzgTOGMLMwwH6oC/mdmHZnanmRVufJCZnWhms8xsVl1d19Z0JbpPGUDePiPoffoeunNJJP2dA6zsZPty0q3RdYqsOv0pGm5+y+sYIrIZ3S3KhhJb1L+xl9jwcSTdEQB2Af7inNsZaKSTD0zn3O3OuYnOuYmlpaVdG3ndDH0CL1865wjNW054Yfp1ABGRDQwDOutf80V8X1ZxztH49w90o5JIGutuUbaY2OXKjU2J79sSi4HFzrl34u8fJlakbbVkzJQB1Ox8Mw3Xvp7QMUUk4dYQm4nf2AhgbYqzeC7y9WpcfavuvBRJY91dXPAX4EYz2x54Lb7tu8Qua3ba5frbOOeqzWyRmY12zn0GfA/Y4js5N5CExaxmRmBMqXqViaS/Z4A/m9nBzrllAGY2GLiSjbr0Z4NQZQ0AwXG681IkXXW3T9nVZtZMbK3GOfHNi4k9hPcvW5HjNOC++J2XXwDHbsVYm0rwTFmwopTWmV8mdEwRSbjfAa8CC8xs3Q96OwAL4/uySqiyGoDgOM2UiaSrbt+G45y7BbjFzHrH3zdsbYh437OJWzvOJtovXyZ22GBFGU3//Ijo2lZ8vbL2aS0iac05V2dmOwNHs35JxC3AA865ls1/Z4byGTm7DcVXnOd1EhHZjG8tyszs+W/Z3/7aOZdWXbLboyW4KguMKQEgPLeOnIlDEjq2iCSGmU0l1hLjLuAuMzsR+BUw2cxOS8QPlD1J0TmTKTpnstcxROQbdGWmbEnSUyRLMrrHArl7DaP/Iz8lMKJ/QscVkYS6CrgIwMxGATcBdwN7A38GTvYsmYhIJ761KHPOJXZ9Vwol6+5Lf1kvCg4bl9AxRSThRgCz468PBV5yzp0Sf4zbQ97FSr3QnBqWH3Yf/e48lNy9O7shVUTSQbeffdmjJKFP2Tqt7y6i+dnPEz+wiCTSur/9k4F1SzGWAFk1zd32STXhz5djWk8mktYyuihL1kwZQMPlM1l91n8SPq6IJMwnwClm9l1gX9YXZUOJPUUka4QqqyHgIzimi423RcQTGV2UJfOhu4ExpYTnrcCFIkk7h4hslXOBXwL/Be51zq1ri3Ew8J5nqTwQqqwhMLpEz70USXPZ8Tc0CTNlwYpSCEcJL1hBcExZwscXka3jnHvdzMqA3s651R123UHscW5ZI1RZTe4e23gdQ0S+RUbPlCXp5ksAAhWxQkyd/UXSl3MuslFBhnNugXOu2qtMqeaiUfL22568A0Z5HUVEvkVmz5QlsSpbtzYjXFUXu69LRCQNmc9HvzsP8zqGiHRBVhRlyVjo7+udy4D3f01gZEnCxxYRSZRoUxuWF8B8GX1hRCQjZPTfUktiSwyAnF3K8fXWY5ZEJH2t+d2zLNvmqqT8cCoiiZXRRVlSF5UBre8tZs2FL+jDTkTSVmh2Df5t+mzwSDwRSU8ZXZQls08ZQGjWYuov+S+RJfVJGV9EZGs45whVVhMcP8DrKCLSBRldlCWzTxnEepUBhKtqk3oeEZEtEV3WQHRlM8HxA72OIiJdkNlF2TpJmikLrmuLMVdtMUQk/bRVxjp/BMdppkykJ8jooizJS8rwDeiF9clTrzIRSUuBYX0p+sM+5Ow0yOsoItIFGV2UJbsqMzOCFWVEvlqVlPFFxFtmdoCZfWZm883s3E72n2xmlWb2kZm9bmZjvci5OcHRpRRfMhVf33yvo4hIF2R0n7JkL/QHKH12Oqa2GCIZx8z8wC3AVGAx8J6ZPdnhGZoA9zvnbosffwhwLXBAysNuRttHSwls1w9fUZ7XUUSkCzJ8piz+axI7VviK8nSruUhmmgTMd8594ZxrA2YAP+x4gHOu463XhST106Z7XCRK7f/cRv3FL3sdRUS6KMOLsuTPlIUXrGDFMQ/R9smypJ1DRDxRDizq8H5xfNsGzOzXZrYAuAo4vbOBzOxEM5tlZrPq6lKzBjU8fwWuJaw7L0V6kIwuyizZK/3jQzfd+yFts5Yk7Rwikr6cc7c450YA5wB/2MwxtzvnJjrnJpaWlqYkV0h3Xor0OBldlKXi8mVgeF/I8cceTC4imWQJMLTD+yHxbZszA/hRUhN1Q6iyBnxGYGyZ11FEpIsyuyhLAfP7CI4qUa8ykczzHjDSzIabWQ4wDXiy4wFmNrLD2/8F5qUw3zcKVVYT2L4/vvyg11FEpIt092UCBCpKCX2wNKnnEJHUcs6FzexU4DnAD9ztnJtjZhcDs5xzTwKnmtl+QAhYBRzjXeINFf1hHyJ1jV7HEJFuyOiiLOndY+NydhpEeO5yXCSK+TX5KJIpnHNPA09vtO3CDq/PSHmoLsrZZZN7EkQkzWV0BbF+piy55yn6/T4M/OR0FWQikhbCi1bT9OAnRNe0eB1FRLohs6uI9oX+adM6SEQk6VpemM+Kn8wgUrvW6ygi0g0ZXZSlak2Zi0apnXo3DTe+mdTziIh0RaiyBssPEtiun9dRRKQbMrooS9WaMvP5CC9YQdtbXyf1PCIiXRGaXUNghzItqRDpYTL7b2wK+pStE6woI6ReZSKSBkKV1eSok79Ij5PZRVkKBcaUEv6sDheNeh1FRLJYpG4t0Zq1BMerk79IT5M2LTHMzA/MApY45w5K0JhA8teUAQQrSnEtYSILVxMYrnUcIuINX/8CBn3xf1ivHK+jiEg3pU1RBpwBVAFFCRsxRWvKAHJ2Hkze90fiWsJJP5eIyOaYz6cfDEV6qLS4fGlmQ4g9ouTOxI4b+zUVHTFyvlNO6XPHEqzQc+ZExDuNf/+AtXfN8jqGiGyBtCjKgOuB3wGbXZBlZiea2Swzm1VX18UF9amsylh3KvVEExHvrL31bZru+8jrGCKyBTwvyszsIKDWOff+Nx3nnLvdOTfROTextLS0q2Ov+96tztkVK372IHX7JHSyT0Sky1w0SmhOrRb5i/RQnhdlwJ7AIWb2FTAD2NfM/pmQkdtnyhIy2refrjBIqLJas2Ui4onIl6twjW0E1Q5DpEfyvChzzp3nnBvinBsGTANeds79LBFjr796mZoiKVhRRnRlM9G6xpScT0Sko7bKGgCC4zRTJtITeV6UJdW6qixFAhWxy6qhqtqUnldEBCCyeA34fQR3UFEm0hOlVVHmnHslUT3KNho44UN2Zt2dl+G56uwvIqnX+9T/YcjaP+Lrnet1FBHZAmlVlCVciu++9A8tpvDEXQmMLEnJ+URENmZ5Qa8jiMgWyuiiLNUdMcyMfn89lLx9R6TmhCIica41TN2B99D87OdeRxGRLZTRRZlXfcrCS9ak7HwiIhBby9ry7Oe4NS1eRxGRLZTRRVmq+5QBNFwxk2VDriS6tjVl5xQRCa2781I9ykR6rIwuylh382UK24YFRsXWk4U/X566k4pI1gtVVkOOX2taRXqwjC7KvJgpC7a3xdAdmCKSOqHZNQQryrCg3+soIrKFAl4HSKoU9ykDCGzfH/w+wupVJiIp5OuTF/v8EZEeK7OLsnVSOFNmOQECI/oRUq8yEUmh/vdP8zqCiGyljC7K1l++TO15i/64L76++ak9qYiIiPRoGb2mrF2Kq7LCn04g/8DRKT2niGSvtXe+R/X4G4iuavY6iohshcwvysxSXpRFm9pofetrfUCKSEq0vb+E8KI1WJ88r6OIyFbI+KLMzFJ++TI0u4baPW6jZeaXqT2xiGSl0OwaguMHtC/ZEJGeKeOLMoyUz5QFx8TaYujB5CKSbM45QpU1BMcP9DqKiGyljC/KYjNlqS3KfEV5+MuLCKkthkiPZmYHmNlnZjbfzM7tZP9ZZvapmX1iZi+Z2bapzhhZvAa3poUcdfIX6fEyvijzolcZQGBMKWE1kBXpsczMD9wCHAiMBY4ys7EbHfYhMNE5tyPwMHBValMC4SgFP5tAzm5DU35qEUmszC/KIPU9MYh19g/NrUv5LJ2IJMwkYL5z7gvnXBswA/hhxwOcc/91zjXF374NDElxRgLD+9H/H0eSs0t5qk8tIgmW8UWZBzdfAtDrlN0o/c8vvDm5iCRCObCow/vF8W2bcxzwTGc7zOxEM5tlZrPq6hI7gx6tb9EPfyIZIqObxwKeVWXBsVrfIZItzOxnwERgcmf7nXO3A7cDTJw4MaEfSLV7305gdAklD/40kcOKiAcyfqYMDxb6A7hIlKYHP6H13UXffrCIpKMlQMeFWkPi2zZgZvsBvwcOcc61pigbAC4UIVRVR2B4v1SeVkSSJOOLMjPAi5l9n7HyhMdo+vsHHpxcRBLgPWCkmQ03sxxgGvBkxwPMbGfgr8QKspTfbh3+fDmEIgR156VIRsj4osyry5dmRrCiTA8mF+mhnHNh4FTgOaAKeNA5N8fMLjazQ+KH/RnoBTxkZh+Z2ZObGS4pQpXVAOpRJpIhMn5NmRd9ytYJVJTS8tw8T84tIlvPOfc08PRG2y7s8Hq/lIfqoK2yBvy+9obVItKzZXxR5lWfMoh19m+65wOia1rwFeuZdCKSWHlTt8fXNx/LzfyPcpFskPmXL8GbNWVAsKIMQJcwRSQp8qZsR9H/7e11DBFJkIwvymJLyrypynL33Y5BX/wfORPV1FFEEsu1hGh9bzHR5pDXUUQkQTK+KPOseyzg65VLYHg/zJ/5f8wiklptHy2jdtKttD6vdasimSLjq4XYQn/vzt/4zw9puPkt7wKISEYKVdYAuvNSJJNkfFGG4emjjpqfqGLtDW96dn4RyUyhymqsMAf/sD5eRxGRBMmCosy7y5cQezB5+IuVuNawZxlEJPOEZtcQHDcA82X+x7hItsj4v81e9ikDCFSUQdQRmrfcswwiklmcc4Qqq9XJXyTDZH5zG68esxS3rqljuKqOnHFa+yEiidH/X0fh65fvdQwRSaDML8o8FhhdAn4fkaX1XkcRkQxhZuTtO8LrGCKSYJ4XZWY2FLgXGEBsTut259wNCRzf08uXvoIchjRepI7bIpIwre8sIrKsgfwfVmAePrVERBIrHdaUhYGznXNjgd2BX5vZ2ISN7vFCf0AFmYgk1Nrb3mHVyY+rIBPJMJ4XZc65Zc65D+KvG4AqIGEt8NOgJqP5ySrqfvgPXDTqbRARyQihyhr1JxPJQJ4XZR2Z2TBgZ+CdTvadaGazzGxWXV3XnyVpfj8u4m0xFKluoOXJKiILV3uaQ0R6PheJEp5TozsvRTJQ2hRlZtYLeAT4jXNuk1XxzrnbnXMTnXMTS0tLuzxubnEBrWsaE5i0+4IVsbx6MLmIbK3wgpW4lrBmykQyUFoUZWYWJFaQ3eecezSRY+f260XLyrWJHLLbAhVlQKwthojI1ghVVgMQHKeZMpFM4/kKdIutVL0LqHLOXZvo8fP69WbtYm8bt/pLCvGVFBCqqvU0h4j0fPk/GsvAuWcSGNbX6ygikmDpMFO2J/BzYF8z+yj+9YNEDZ7fvzd1H33Jwhc/+tZjnXNJW4yfu9cwLM/zGlhEejjz+wiOLtVd3SIZyPOizDn3unPOnHM7OucmxL+eTtT4uX0KAXh06oUAtKxsYOELH3Z67LuXPcT1/h8RampN1OnblTz2M/redEjCxxWR7LLmwhdofu5zr2OISBJ4XpQl26if7N3+OtTYwuP/ezGPfv+PnRZe71/9GABt9U0pyyci0lXR5hD1l75C21tfex1FRJIg44uywf8zhv996BwAZuzxO5a9/RkAkdbQJseG44VauKWtS2O/cMLNXbosCtD20VKqd7yR1jcXdul4EZGNhT+thajTIn+RDJXxRRnANvvuCMDyT75q39ZZ4RVpC8f2dZhFa6xexarPl2xybDQSYfadz7dfFv02vj75hCqrCc2u6U50EZF27Xdeqh2GSEbKiqIsr19vxp90wAbbIt8wG9bx0uZdw47nntGnbHJMNF7AdZV/m2KsIKheZSKyxdoqa7C8AIHt+3sdRUSSICuKMoBtvrfjBu/v3u5EWlY2dHpsx5myzi5zwvpZta4yn4/A6FL1KhORLRatXUtghwGYP2s+ukWyStb8zR524Hc22bbq8yU0r6in8o7nNtgebmrFOcf71z6+2fE2V6x9k2BFqXqVicgW6/+PIxnw5klexxCRJMmaRjc5vfLJ6Z1PW0Nz+7ZZVz1KqKmVhc99SPneO7RvDzW1snjmbF49++72bbUfLqB0p+GYL1bHdnemDCD3eyPAZ7hotH2cdOOiUVpWriW/pMjrKCLSCcvJmo9tkayTnpVBkkyfdxvfveaX7e/nP/Y2DQtjlxPrF66fwQo3tbLqsw0X99+3y5ncMfSXvHjyrcCWzZT1+uVE+v/jyLQtyADeuugBbiv9GU21eni6SDppfWcRdYfcS2j+Cq+jiEiSZNWPXIUD+jJ4z4oNtq2cuxiAxw64qH3b5w+9wRdPvrvJ9zcuXUnlX5+lbJftGLzn2PbtzjliT4v6ds45CEexoH8LfgfJN++hNwBoqqunoKyPx2lEZJ22dxfR8tRcfLf9yOsoGaG+vp7a2lpCoe7/gC3SUTAYpKysjKKirb/ClFVFGUAgL+dbj+msIOvoy//M4qWTbm1/H25uI1iQC8CCJ94mt28vhnx33CbfF20LsbT8SnqfvBvFl0ztZvLUcFEHgPm6VmSKZDIzOwC4AfADdzrnrtho/3eB64EdgWnOuYeTlSVUWYOvXz6+Qb2TdYqsUV9fT01NDeXl5eTn53f5h2qRjTnnaG5uZsmS2NW1rS3M0vc6WpL4vmWGauz0733rGBsXbW0NTbxz6YOsqFrEkz+6jIcmn7/J9yx44m1e+tVt1C9fxYKbXuhe6LhQYwtrvkpun7NoJAKsL85EspWZ+YFbgAOBscBRZjZ2o8O+BqYD9yc7T6iymuD4gSogEqC2tpby8nIKCgr05ylbxcwoKCigvLyc2tqtv5Ev64qydQv0+4/bll+tfmCT/aU7DqNk/LbdGvPrFz/mzT/8k/t2/s0G2+c+MJOZZ9/FvEfe5MkfXcbsu16ggVZy1nT/JgGARw+4iLuHn7BF39tVLhJ7IPuWrJnbnCcO+RMz9vxdwsYTSZFJwHzn3BfOuTZgBvDDjgc4575yzn0CRJMZxEWjhGbXEByvTv6JEAqFyM/P9zqGZJD8/PyEXArPuqKs3+hy+u+wDfvefBK5xYX8YMZvN9i/dtlKfvLmVd0a89mfXQtsWMiEm1t55qfX8MG1T/DvH6+/4tFAK73J4fH/vZgFT62fcat+bx51H3/Z/r5lZcMmi+2Xvv4psH42q/q9ebx86m2xdWpxoaZWZl39GJFQmJd/fVv7mrmuWjdDlsii7Iun3mXZm3O3epwPrnuC5bPXP6Yq0hbiv2fcQfPy+q0eW6QT5cCiDu8Xx7d1m5mdaGazzGxWXV33exW6Na0EdxxIzqShW3J66YRmyCSREvX/U9YVZYH8XH4x+2aGTI6t+dpmv5022D9g4khyeuVzWtND7HvryVt8ntXzl3W6vZ5WfPhoenouL554CytmvM+KEx7lo0lX8MaEi3hl94tZ/cwc/tr/Z9wz+hScc5s8Emr+Y28D8OBe5/DxLU9zve+H7QXcWxfex2u//RvvXvogH9/6NE8dfjkAbWubaaxeRVPdGiDW4uP28unt79dJxkzZOp09BL6rWtc0MvOsu3jswP/Xvm3+Y2/z0Y1P8epv/5aIeCJJ45y73Tk30Tk3sbS0tNvf7+ubz4A3Tqbw5zsnIZ2IpIusK8o2llNUAMD3/vorTqr9BzhbY/cAACAASURBVKOP3AuIFW87nfIDfj775i6Ns83UCRu8f2jK7zs9bgn1fEotq2mmqXoVrx51A6vveY8dGMB3KGfEOy00/OA+cgnQurqRN8b9kc96nU/t/n9jIuWMYwCfHXEH9fOribSFCeDDj1H97jw+vu0ZlrwWm02r+yg26xZa2wLALb1/wu2DjuGvZT+neUU97/zpQRqXruST255h7dL1t9h3nClz0Sgtq9f+//buPC6qqn/g+OcwICCbgOK+oYZprphLuWXlVmguuVaYlWa22PIrqyezp9WectfKXUorLUvNNDV3c09SwQ3UFFDZVPZ1zu+PexlmYAZQgUE979drXlzOXc45M5fDd84999xS64ma7fb4De0XteMYCeFah4WDY/4pW1YBZMLxC6wd/FmJH0x/K4naGWbRQ1uUaaIfu9/9tsTHzsnMZprox+GZa62uv7j3BNNEPxKOX7C6vgKKBsy7puroaYqiKGXijrv7siCDkyOvyjU211dtXo/gE3NZHfQRV0/H0Om/I/Br24hNo2eSFpvfy1RwYLzNRzhhJJz8wYDnuMq5nKsIwAVHXHDCBUcy0AKNi+FncKIKPnEp+OGOC44ItH+uAPdQncb4kjvoVxyzMriLbOpSmwOr9wHgngIZWyLxwJkMssnGyJKAcdTs1BSAPZOWs2fSctN7IPVLo4knokzThFRpUounT31tKvNf7y/n0Be/8FLqyiLf26R/Y7l6OgaBwIDAAUFuTBI4OxL/70XcanrjdD6VkyFbOb/+MA/OHIvMzMGxkQ+VWtdCZudy/rVVhM5eh2e9ajTGB098ydxzHudO9SAzhwCqUi0sjaSPt2qfgVHi8lBjshu6s7T2s/R7YjAetX1BAkaJNEoqD2qO8331yblwleTPtpObmY2DoyPo++/5aw8Rx09yaWA3XH+/AEYjSP0zFuDxehec761D9sk4Upf+jXBxRLg6aS8XR1z7BmCo5UnupWSyw2P1dY6m7Qw1PRCVHG1OpZJxNQWXKu4ApMVeJSspjSqNaxX5XpfUyq5vAzA29lsMzk44619KCsrN1sY97v9kJfd//GSRx1w39HNOrdzN2MshAOx5fzltXg4qtN2J73cC8O8ff+N79y1xGe4A0EQI0RAtGBsGjLBHQRKf+RljQhpVfy36s1AU5dZ2xwdlJeETUIcBGyYTOus37nnmYdxr+RL4fwPZ+X+LefTnifw26DNaPt+bC3/+c8N5SCCdHNLR/hk2HtiJiFV7iCKJKJL46/B507aVMJA1WgvKokkinWxcspxwxREXHHHH2bRtw0RX4h5cSC+aAJCLkSsJ6RyJvASAP95UwpEjj86mYe9AGiS7kQJsn7AAgACqUvm0JPGZn7l2MoZLu0+QRAY5ZCKNRuIf+47cqCRkZg5Xwy/g6uWO55DW+MwbwLLAV+mZUIdB5D8tIab2Z1QeHcjyRUvxqFOVXtE18JbgjSvxQdo/9UrPtGH5wm8Z8Ot/MMw+TFtqafe4UQvOQcb6kzh3qofINNKCGnAkg2tH8u9oFa5OXIjIxgkDxuVhJDvqd9waBMLggFPTaqRXr4TxVAKp3/9DxpUULUDxcUc4CByk1kOWHZeCYe95EAIc9JdRIpO03secyESSv9gF2bkWn2X2iiF4P96ak2+uxOPbiEKftd/+F3C+tw6pCw5yZdxqPVjTAraszCw2xoYy5PQsKoUmED50ATnGHJqM6GoK6rw+ehiHKq6k7zhD1v4oHFydtGchOggQArdRbRFOBjIPRJFzIs5UduEgqI0n0STxjd+T1PCpxmOL38ivm4NAODng8mBjspPT8cSZShjI3HkWHLTjC2cDldpqw6qyIxKQKZlcXnEATyqR+c9F3HAiO9fGmPe8sY+3yFgeKWWOEOJF4A+0KTEWSSnDhBD/BQ5KKdcIIe4FfgG8gSAhxAdSyuZFHPaGZO67gGNDn9I+rHILGjVqFEuXLgXA0dERLy8vmjZtSr9+/Rg/fjxubm52LqFyM1RQVkJV/GvQfdqzpt8DX3+M5qMexLWqJ6/KNRaD7es91IrOU0bhUceXeTVHIY3Xf2OWz911gT1W12WRHwTEkUocqTaPE8pFYv2dyDwTr/fEOZKLJFF/BmddqlANN1gXw9V1MTTDhwRcOI/WC1gLT9ypRMYfp8mOvYonzqb8s1MzcKjiClJy7UIciaSTey2Ntm21Xp2MhGROEodAYMSIEYn/4PvwGng3LILkqHh2kYERiRGJNAiGHvySc/uPw0I4sngTUZwAJHnvbu3u99DnpQ6c/vkvwn/dxXnCqPNACxr0bkPrCf1wcHJECEHWzLWkkkXk8/X4Z+7vpvcjr0dwmugHwCMr32Ld41MgE547sBj32r6s99fucE31gKrbRuFe29dqj5Zr3wDqZn2IzDUiM3KQ6dlEbzpCyJD3aRr8AJHfbscDZwav/y8yPdu0jaO/9s/VqXVNPN7qakqX6dlcXHOAHIwkhJ0n7OlF1DdWxgVnsvZdMG3nOVmbtuXYs0upcbrwpVuH3v7ISg5kLQslZcZfFus6Upef0QL62omViO9f4PKkmxNe4S8ijUbuxo+6eBHbdb5ptaGOF9kLe5ISk4jh/d24nc/gYT3gT3k4hPupz5ac8xTlVhpgLaX8Hfi9QNoks+UDaJc1y64MWTnknIzHtd/dxW+s3BG6dOnCihUrMBqNJCQksGvXLj799FMWLlzIjh07qF79zrxLNysri0qVip+LtCK748eU3SghhMXzIc3/0Xg1qkn1to2o7FeFOt20L8095j5P4wEdufvJBwAIWvU2jfp3sHl887FTNyONbCLORHKBa5wmgaNctrh8up2zrCKM3zjBGo7zK+FsI/8u0K2cYS0n8It8nV31EtlIBH8TA0DSv3Ecdo3lu99+Zu0/O9hPFIeIxv35/HqdJJ4TxHGKBCJIZONPv5Hknh+kXiKFWFKJJ42E3FRya1XG6Kb1bEWu3kcmOWSSS5b+ynGQrB06hd8Gf8a5Pw5jRHJ+6xF2vLWU6J3hps8hPUG7fOxYOb/X0Jp1j08xLV85rdXLyc0F0KY6WVB3NKGz12ll3X+KTc/OMt39eu3sJe0SpMEBB7dKGKq6cfl8DEYk4Uu3kEku8aSxb8N2uL82bsNb4T66HQZf7ZKh8711qPJxT7y/7IvP3P74Lh7MIefLZJLLpQMRRKZcYgtn2EQENU69Rvijnuxrl4OhqvZNePfpw6wmHL9zb+C2ZzR+Z16n5oW3mNfgGebVCMbzPw9Q4/Tr1Dj1GjVOvEqN8AlsIr/nLpxYnH8cSPUDL+C3bxzV/hrLltQTfNfyZbKT0zlOLNs5S7XNo6m28WmqbhhF5em9WdXrfTY+PYN958PwXTWSvzjPHs6zl/Mc4ZLtOe70Ly7/zPnd+nrFquwTcZBjxKlFDXsXRakgKlWqRI0aNahVqxYtWrRg3Lhx7Nmzh7i4OCZOnGix7axZs2jatCkuLi40adKEjz/+mJwc7YrMu+++S0BAQKHjjxs3js6dO9vMf9OmTXTv3h0fHx+8vLzo1q0b+/dbjlVNSUlhwoQJ1K1bF2dnZxo0aMAnn3xiWh8bG8vTTz9N9erVcXFxISAggEWLtOdNb9u2DSEEUVGWswc4OjqyZMkSAM6dO4cQgmXLltG3b1/c3Nx47733kFLy3HPP0ahRI1xdXfH39+edd94hM9PyRrPNmzfTpUsXKleubKpDZGQk27Ztw2AwcOGC5djXkJAQvLy8SE213QlSGlRPWSkafuBLvr/3dVMgBnDPsz25sPUojQd2otW4vkgpaf/u4/gE1MG/X3vTMzZDmr9ocaymI7px+VAkrcf3tbjjsCwYkWRQ9Nxp5zeFknbZ8k7Nb1u8ZHXbP8fNNQVF1pxZd8DmuuQL8RYPjS8oOzWDS/tOWV0XF3qG9PgkZK6RlAvatAOpMYlWj2HN1dMxOHtV5toZ7dJu3OEzAET8/BetX3yEX/p+QEZCMscWbqLpE9058d027v/0KQKGdiYh7AL+j95r9biHZ6zlxLLtpMcn8cy5BXjW9zOt+3dzKHW63cOh/62i1v3NyLym/cEXHJOYmZTG0W82aOVPy8TByYAEsjGSEBPLj/e9SfPRD9HmlSBTz+yxFdtp9UJfspLTyEnPorJfFZLIb5jSyWbZ0P9qPb1GI78+8l8SSYdrWn7atpm4PNjYtE9qWH4vWCLpOD7UkBgK3AiSCSeWb6fpiG4WyXnB2pVT0SRHxeNRp6rV90uxlH1UmzBazVFW9mK7zy+U5jqkBR4vdMSYlkV836WF1ruNaovbqEBy41NJGFx4DmH3cR2oPLQlOReukvhk4XG4fttKZ+7J2rVrM3LkSEJCQli4cCEODg5MnjyZxYsXM336dFq3bs3x48d5/vnnycjI4MMPPyQ4OJhPPvmEffv20aGD9mU6MzOTH3/8kc8++8xmXikpKbzwwgu0atWKnJwcpk2bRu/evTl9+jS+vr5IKXn00Uc5f/48s2bNomXLlkRFRXHy5EkA0tPT6datG66urixbtgx/f38iIiJITCzcXhfnrbfeYsqUKcyZMwfQZtj38/Nj+fLlVK9enSNHjjB27FicnJz44APtf+nmzZvp1asXL730ErNnz8bZ2Zndu3eTnZ1N9+7dadKkCYsWLeL999835TN//nxGjBhR5peHVVBWimq0a8Lz8d/h6pvfg9Z0RDeLf05CCHwCtKsdDgYDvs3qAVD3gRZc2HrUtJ33XbV5bO17XD6k9WzUe6gVD8wey/5PVlK1RQN2FpgGwrO+n+mh6u3eHMjBz1eVSp36//Yeqx/9kNVBH5Z4nyNfbyiUNurkV2weO5eobUc5+9vBQusbPtKOs+sOcu3MJbaM+8pi3T3P9eTY/I2A9rB45ypuZF4t/G3l8MzfSD5vOQfUieXbC223bmjheegMzk5cORXD5jFzTGl5c7xFbT/GN35PkmEWaJ74bhsAu98OYffb2li44ONz2TWxcKMNmO5gvbDlCM2ffgiA+KPnWPXwJOr3bMO/Gw/jWtWTHH3akASz+dgAUqLzG6uCd7HmzV8XtmgzYYs2m9K3jP+aGh0DWNPvI1KiE3gl+xerZYsPO8+391h+KVjR2fLbdnJUPOuGfI5vM8sB+mlxSQiDg+lO2DzrR35Jk8H3YajkZEozv8S/oO5oWozpxUPfjLdaJiWfoaYHrkNa4BRw/VNpKHeW5s2bk5SURHx8PO7u7nz++eesWrWK3r17A9CwYUM++ugjXn75ZT788EPuuusuOnToQEhIiCkoW7t2Lenp6QwZMsRmPgMGDLD4fd68efz8889s2LCBkSNHsmXLFrZv386BAwdo164dAP7+/nTt2hWA5cuXc/bsWSIiIqhTp45p/Y0YO3YsI0eOtEj7+OOPTcsNGjQgMjKSuXPnmoKyDz74gD59+jB9+nTTdk2bNjUtjxkzhhkzZvDee+/h4ODAiRMn2LVrFzNnzryhMl4PFZSVMvOA7Hp4Na5pCsqc3PNnmvZr40/HycNp+Xxv3Kp703vpq0ijkaRzly0uAw3dPYW4f85S876muFRxL3FQ5uVfAwdHA1dOWb/Tv0HvtjToE8i59YeA/BsQ8lRt2YCOk4ax6bnZZF5JKbR/k8fvp9uXo/GoW43Bf37IdMNjJFqZEuHeiYOJ2h5G+NIthdY16t/BFJRlp2ZSycO1UFBmcKlUKCCzJnLtfs6uKxwUevnX4NAX1oMWoETTghz5pnAwWtDG0TMJD9lK1Laj+LVtBMC/Gw8X2i5vapM88UfOFtomz843l9hctzzwVdPyDKcBVrcpGJAVdHjmWra9ovUgXNxjOQlw9I6wQgFZnisno6naogFpcdfY8sJXpMdb9v7FFwg8FetcejTCpUcjexfjjlBUr5VD5UpFrjdUdStyvWPdKqXWK2ZL3hcfIQRhYWGkp6czaNAgi+E1ubm5ZGRkEBcXR7Vq1QgODua9995j+vTpODk5ERISQr9+/ahSpYrNfM6ePcukSZPYs2cPsbGxGI1G0tLS+Pdf7W/60KFDeHt7mwKygg4dOkSzZs1MAdnNaN++faG0+fPns2DBAs6dO0dqaio5OTkYzcZ2Hzp0qMiewODgYN59913++OMP+vTpw4IFCwgMDKRNm7KfJ1CNKasguk97lsFbP2bIzs8YdSL/YefCwYFO7w/Hrbq3RVrX/z1t0QPnXtuXhn3bmaZSCFr1NkG/FH4Gp7kBGyYzOnIeVVs1ACBgeFdqdgzgqbD8udkcDAaCVr1t+t08z3ZvDeLJf2bSZNB9eNTxtZmPR91qpnIX1O6tQfSY+zy1OzejQZ+2hQKmkYenU++h1tTv2QZnLzfS466RnZqJs7e7aZvA1x+j1v2Wg6B7zLE+8e/+j1ZYTa/bo4XN8pfU4em2p1YxF7VNC75j/460SC8q8Fs/4ksAHF3LfxBrXkBmzcanZxRKy5t0+duWLzNN9OMbvyc5/dNfpnrnMR+TqdiWm5hm7yIot4iwsDC8vLzw9fU1BSErV64kNDTU9Dp69CinT5/Gx0e74WjYsGEkJyezbt064uLi2LBhA8HBwUXmk3dpcs6cOezdu5fQ0FD8/PzIyiqduR0d9P8V5r3rubm5FoFVnoKXE1euXMn48eMZOnQov//+O4cPH2bSpEnX9QgkX19fBg8ezPz588nKyiIkJIQxY8bcYG2ujwrKKggnNxfqdm9B7c7NcK9tO8DJ4+jqTJ9lr9P1y9E8MHtsofWNB3SyGOP0wtXv6f3da6a7Dz3r+9GgV1tAC2Du+3Akfb57jWF7/odvs3q0fbU/3aY9o+Xlkh8INBl0H89eWMTgLR9x/8dPWJTfXOAbWq9MwScmDN//pWm56chudP70KVqN61sonzxe/jVwdHZi4B8f0GHSULKS08lITKbFcz15JfsXXkxZQdcvRpuClbo9WjLuynJavdCXIbsKfxO6tP8UAcO7UruL5XOle8x+3iLQM/fgNy8A4OBo+TD7xgM6Wt2+uJsL3OtUNb0/1gSbBeUF5T2Jwpa6D7TAv1/hb443ouBnV1IN+gSWaDsVlBXPeC2DGN+PSJ6+295FUSq46Oholi1bxsCBA3FwcKB58+a4uLhw5swZGjduXOhlMGjtmbe3N0FBQXz77bd8//33+Pj40KtXL5v5JCQkEB4ezsSJE+nVqxfNmjXDxcXF4mHcgYGBXLlyhYMHC1+VyFsfHh5eaCB/Hj8/bdxtTEyMKS00NNQiSLNlx44dtGnThtdee43AwECaNGnCuXPnCuW/cePGIo8zduxY1q5dyzfffEN6ejrDhw8vNu/SoIKyW1zga4/RevwjVtc5OBpwreZF1y9H4+zlxt0juwPwYsoKi3/8lat50eE/Qy16srpNfYa2E/KfvfzIijcZvu8LADzqVKXuAy1xMOQHKR3fH2aRd5cpwYw4OJWWY3pbpFdv15jANwYw4tA0+nz3ukW3eoaVmwPMJzf1bJA/QL6ShysOjgZTMNhj9liaj36Ivt+/YeotrH1/Mx5Z+VahYzYd0Y3OnwVTpUkt/Pu1p9dS7UHyRrP5xpy93XH20r6BNerfgRrt76LnklfwqFcNJ3dXxl4OIWjVO7yU/lOh4OWl1JWMS1xOj6/GFcrbyd2Vp099Rd0H8nvmRv49jcaD7tPq5VmZKo1q8sjKt2j35kB6LnnFYv+2r1k8D9uk5n1Nuee5ngze8jE9zIL0FmN7M2RnfnDab81/cHBypO2r/S16RK1xrOxMp//mz5Vq8dQKIajSpPCEto/+NBGvBrYHpPeY+zwN9S8L1oJwxVL2MW2Qv2Pj4r+oKXeOrKwsLl26RExMDEePHuWrr76iU6dO+Pn58emn2qP13N3deeedd3jnnXeYM2cOJ0+eJCwsjB9++IG33rJsF5966il+++03vv76a0aOHGkK2Kzx9vamWrVqzJ8/n1OnTrFnzx6GDx9u8YD3Hj160KVLF4YOHcrq1as5e/Ysu3fvZsECbf7L4cOHU79+ffr168fmzZs5e/Ysf/75Jz/++CMAjRs3pn79+kyePNk0nuvVV18t0XQ6AQEBHD16lNWrVxMZGcmMGTNYtcpyOM97773H+vXrmTBhAkeOHOHkyZMsWbLEdCMCQOfOnQkICOCNN95g2LBheHh4FJt3qZBS3nKvwMBAqVQ8udk5cipBcipBN7T/hR3H5Cz3IXLHm4vlH6NnyD/Hf2WxPuH4BdPxd70TUuLjrhnwsZxKkFx272ty5YP/kTmZWVa32/fpStPxV3R72+o22RlZMjs90yIt9p8zpv0K1t08fSpBMi3+mpRSysykVFNaVkq6PPC/VVq+3d8plGfedomnoqWUUqbGXpUxe47LqJ1hcufbS+WJH3ZYbJ+TlW3aZ+sr86TRaJQzXAbJra/MK1SXguWLWLPPtHx04UYppZQ/PfQfOZUg+ffMtXIqQfLM7wdlbk6OPPP7wUL7ZyanSSmlvPZvbKF1cUfOSimlPLfpsFa2CfOtvse2oE3Yavf252Zf19N+JX+1V57nbZl9LrHkb5RSrPDwcHsX4YYFBwdLtPnGpcFgkD4+PvL++++XU6ZMkSkpKYW2nz9/vmzVqpV0dnaWVapUke3bt5dz58612CYrK0tWq1ZNAjI0NLTYMmzbtk22bNlSOjs7y7vuukv+9NNPslGjRvL99983bZOUlCRffPFFWaNGDenk5CQbNGggP/30U9P6ixcvyieffFL6+vpKZ2dnGRAQIBcvXmxav3fvXtm2bVvp4uIiW7ZsKXfs2CENBoNpm7Nnz0pA7ty5s1BdxowZI729vaWHh4ccPny4nDVrltTCnXwbNmyQHTt2lC4uLtLT01N2795dRkZGWmwzffp0Ccj9+/cX+55IWfR5VdL2y+4N1I28VFBWcd1MUFYcY26u6fi/j/iixPvlZufIrJT0Em2bnZYhf+41yRRAlFRq7FUZtvRP+feMNRbpB79YJb+pFSyTYxLk1TMXLdb9MXqGnEqQNBqNMmzpn3IqQfLXoA8LHTuvzulXkktcnq0T5supBMmL+05KKaU0Go1Wtzu5Yqfc+9GPpjyuRMTIqQTJOV7DTNv82HWinEqQ/PdPy8Y641qqXHL3C/LCtqOm/Y25uab1OycukWsf/0we/GKVjFi915RuNBrl6V/2yPSEpBLXR8o7MyhLfOFXecFzss3PT7kxt3JQppSf//u//5OtW7cu8falEZSpuy+VUtVr6QR8mpbNBOfCwYHBWz5i68vzaf8f27drF+TgaCg0HswWR1dnBm64/nnhKlfzotlTPQqlB74+gMDXrY8fe3jBSzz41TiE0B7/BGBwcSq0nV8bf2IPnzFdTi2J7tOetXgCha1u/7se70xGYjJHvlpP65cfpUqjmoxLWIaDU/77VbtLc6J3hOFZz3JKBmfPygSHa1OIPLZuEieWb7e4BN75U+uDhYUQNH7M+ng8xVL2scs43VP9lnoKgqLc6q5du8apU6eYN29euUyDYU4FZUqpshaYlKa6D7TkqaOzyjSP8iKEMM3jlTf/l39Q4UH6g/78iOTzcWX2j9nFx4PnohZb/G6u0+Th3P1E9yIfit6wbzsa9rV++7ty49zHdQCDGvqrKOWpf//+7Nu3j2HDhvHEE08Uv0MpUkGZolQAfm0aMfZyCJX9Cs8N5OLtjouNO0PLg4Ojocx6P5WiVR52Y3fAKopy47Zt22a3vNVXMEWpIKwFZIqiKMqdo0IEZUKI3kKIk0KICCHExOL3UBRFUZQbp429VpTSUVrnk92DMiGEAZgD9AGaAcOFEM2K3ktRFEVRboyTkxPp6en2LoZyG0lPT8fJqfCNWtfL7kEZ0B6IkFKekVJmAT8A1mfIVBRFUZSb5OfnR3R0NGlpaarHTLkpUkrS0tKIjo42PYngZlSEgf61AfMnVEcBHQpuJIQYA4wBqFevXvmUTFEURbnteHpqj/iKiYm5rmciKoo1Tk5OVK9e3XRe3YyKEJSViJRyHjAPoF27duqrjaIoinLDPD09S+WfqKKUpopw+TIaqGv2ex09TVEURVEU5Y5REYKyA0ATIURDIUQlYBiwxs5lUhRFURRFKVd2v3wppcwRQrwI/AEYgEVSyjA7F0tRFEVRFKVc2T0oA5BS/g78bu9yKIqiKIqi2EtFuHypKIqiKIpyxxO34hwtQog44N8Sbl4ViC/D4lRUqt53jjulzvWllNXsXYibdZ3tF9j/81X5q/xV/jevRO3XLRmUXQ8hxEEpZTt7l6O8qXrfOe7EOt9J7P35qvxV/ir/8stfXb5UFEVRFEWpAFRQpiiKoiiKUgHcCUHZPHsXwE5Uve8cd2Kd7yT2/nxV/ip/lX85ue3HlCmKoiiKotwK7oSeMkVRFEVRlApPBWWKoiiKoigVwG0dlAkhegshTgohIoQQE+1dnpslhDgnhDgqhAgVQhzU03yEEJuEEKf1n956uhBCzNTrfkQI0dbsOMH69qeFEMH2qo8tQohFQohYIcQxs7RSq6cQIlB/HyP0fUX51tA6G/WeLISI1j/zUCFEX7N1b+t1OCmE6GWWbvW8158vu09P/1F/1qxSQZVV+yWEqCuE2CqECBdChAkhXtHTS+1cK0EZ7NaWCSECzOoYKoRIEkJMKMv627tNs5H//4QQJ/Q8fhFCVNHTGwgh0s3eh6+Ly8dWXYrJv9zaNhv5/2iW9zkhRGhZ1f+6SClvyxfaczQjAX+gEvAP0Mze5brJOp0DqhZI+xyYqC9PBKboy32B9YAAOgL79HQf4Iz+01tf9rZ33QrUqSvQFjhWFvUE9uvbCn3fPvaucxH1ngy8YWXbZvo57Qw01M91Q1HnPbACGKYvfw2Ms3ed1cvmuVBm7RdQE2irL3sAp/TzqdTOtRKUoUK0ZXodLgH1y7L+Nv62y61Ns5F/T8BRX55iln8D8+0KHMdqPrbqUkz+pfZ+U0zbZi3/Auu/BCaVVf2v53U795S1c/6fEQAABulJREFUByKklGeklFnAD0B/O5epLPQHlurLS4HHzNJDpGYvUEUIURPoBWySUiZKKa8Am4De5V3ookgpdwCJBZJLpZ76Ok8p5V6p/QWFmB3LrmzU25b+wA9Sykwp5VkgAu2ct3re69/oegA/6fubv4dKxVNm7ZeU8qKU8m99ORk4DtQuYpfrOtduomj2aMseBCKllEU9YeGm62/vNs1a/lLKjVLKHP3XvUCdIt4DisnHVl2Kqr8tpd62FZW/vv8Q4PuiCnUz9b8et3NQVhu4YPZ7FEU3PLcCCWwUQhwSQozR06pLKS/qy5eA6vqyrfrfqu9LadWztr5cML0ie1G/xLDIrFv8euvtC1w1a4RvhXrfycrl71QI0QBoA+zTk0rjXCuJitKWDcPyn3F51R8qVps2Gq3nJ09DIcRhIcR2IUQXs3LZysdWXYpTEdq2LsBlKeVps7Tyqn8ht3NQdjvqLKVsC/QBxgshupqv1KP3236OkzulnrqvgEZAa+AiWje7otw0IYQ78DMwQUqZRPmea3Zvy/RxR/2AlXqS3f7W7NmmCSHeBXKAZXrSRaCelLIN8BqwXAjhWdLjXUddKkrbNhzLwLy86m/V7RyURQN1zX6vo6fdsqSU0frPWOAXtO7cy3q3al73aqy+ua3636rvS2nVMxrLbvoKXX8p5WUpZa6U0gjMR/vM4frrnYB2GcSxQLpSMZXp36kQwgktIFsmpVwFpXquFauCtGV9gL+llJf1spRb/XV2b9OEEKOAR4GRejCBftkwQV8+hDaO665i8rFVF5sqQtum7zMQ+NGsXOVSf1tu56DsANBEvyujElo39Ro7l+mGCSHchBAeectogzSPodUp7y6cYGC1vrwGeEpoOgLX9O7VP4CeQghvvbu4p55W0ZVKPfV1SUKIjvpYgqfMjlXh5P2h6wagfeag1XuYEMJZCNEQaII2CNXqea83uFuBwfr+5u+hUvGUWfuln/cLgeNSyqlm6aVyrpUg/4rSlln0kJRX/c3YtU0TQvQG3gT6SSnTzNKrCSEM+rK/Xt8zxeRjqy5F5V8R2raHgBNSStNlyfKqv03yBu8QuBVeaHexnEKLdN+1d3lusi7+aHeb/AOE5dUH7Xr6n8BpYDPgo6cLYI5e96NAO7NjjUYbPBkBPG3vulmp6/doXcjZaNftnynNegLt0BqASGA2+pMt7P2yUe9v9Xod0f/wa5pt/65eh5OY3W1l67zXz6H9+vuxEnC2d53Vq8jzoUzaL6Az2uWVI0Co/upbmudaMfnbvS0D3NB6WLzM0sqs/vZu02zkH4E2RivvHPha33aQ/rmEAn8DQcXlY6suxeRfbm2btfz19CXA8wW2LfX6X89LPWZJURRFURSlAridL18qiqIoiqLcMlRQpiiKoiiKUgGooExRFEVRFKUCUEGZoiiKoihKBaCCMkVRFEVRlApABWWKoiiKoigVgArKlApFCFFHCCGFEN3tXRZFUZTrodov5WapoExRFEVRFKUCUEGZYhdCiM5CiN1CiGT99Y8QohfaDNMAW/VvnOfM9nlY3yddCBEthFgshPA1W79ECLFZCPGqvj5NCLFSCOFTztVTFOU2ptovpayooEwpd0J7COwaYB/QVn9NBtL0ZdAedVETuFffpwfa88R+AFoCjwENgFX6c8jytAceAHqjPZKjNdpz/hRFUW6aar+UsuRY/CaKUuo8AG+0h8me1tNOgzYmQ/89UUp5yWyfScBMKeWsvAQhRDDwL9AK7TlloH3ReFJKeU3fZjzwhxCisZQyoqwqpCjKHUO1X0qZUT1lSrmTUl4BFqA1NuuFEBOFEAHF7HYvMEEIkZL3AsL1dU3MtgvPa9B0u/WfzUql8Iqi3NFU+6WUJRWUKXYhpXwOCAQ2Ad2AY0KIsUXs4gBMQevON381AdaXbWkVRVHyqfZLKSvq8qViN1LKY8AxYKoQ4mtgDPCLvtpQYPODQPMSdOHfLYTwlFIm6b/fp/8Mt7WDoijK9VLtl1IWVE+ZUu6EEI2FEFP0O5jqCyE6AV3QGp54IAXoKYSoIYTw1nebBPQXQkwVQrQWQjQSQvQWQiwUQriaHV4CIUKIe4QQXYE5aGM/1HgMRVFummq/lLKkgjLFHlLRuu1/AE4BPwN/AS9KKY3AeGAIEAUcBpBSbgV6oN25tBM4AkwDkoFss2PvB3ahXVbYABwFRpd5jRRFuVOo9kspM0JKae8yKEqpEEIsAepIKR+yd1kURVGuh2q/FFA9ZYqiKIqiKBWCCsoURVEURVEqAHX5UlEURVEUpQJQPWWKoiiKoigVgArKFEVRFEVRKgAVlCmKoiiKolQAKihTFEVRFEWpAFRQpiiKoiiKUgH8P2pr9ji6xkAfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from nndl import plot\n",
    "\n",
    "plot(runner, fig_name='cnn-loss4.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "*******\n",
    "\n",
    "在本实验中，使用了第7章中介绍的Adam优化器进行网络优化，如果使用SGD优化器，会造成过拟合的现象，在验证集上无法得到很好的收敛效果。可以尝试使用第7章中其他优化策略调整训练配置，达到更高的模型精度。\n",
    "\n",
    "*******"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.5.4 模型评价\n",
    "\n",
    "使用测试数据对在训练过程中保存的最佳模型进行评价，观察模型在测试集上的准确率以及损失情况。代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Test] accuracy/loss: 0.7234/0.8324\n"
     ]
    }
   ],
   "source": [
    "# 加载最优模型\n",
    "runner.load_model('best_model.pdparams')\n",
    "# 模型评价\n",
    "score, loss = runner.evaluate(test_loader)\n",
    "print(\"[Test] accuracy/loss: {:.4f}/{:.4f}\".format(score, loss))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.5.5 模型预测\n",
    "\n",
    "同样地，也可以使用保存好的模型，对测试集中的数据进行模型预测，观察模型效果，具体代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The true category is 8 and the predicted category is 8\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAJIAAACPCAYAAAARM4LLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAFdFJREFUeJztXVuMHFda/v669GV67uOx49hmN4mTbByWBMiGrOABLUSKQCI8ILRBQou0Ei8ggcQDq30BJJDCC/AGikREHhAhEkhEaCVYZYMIl11y3WQ3ie3Yie2xZ+zx3PtaXVWHh+70f3Fm3OOpbdvj80lRTvU5XXWm/Pf57/9Pzjl4eOwVwc3egMf+gCckj0LgCcmjEHhC8igEnpA8CoEnJI9C4AnJoxDsiZCI6CkiOklEHxHRN4ralMftB7pRgyQRhQBOAXgSwAKA1wE845x7v7jtedwuiPbw3ccBfOScOwsARPQigKcBbEtIE1Ozbu6uI70LQ79Z2h2M8zxXc+VKeTAOw3AwJpBaF4hLIj1H24wd9LNCcRN9BwD02XNZlup9yD1es49r7ir24sS6HSAm80zvXz4vCJjh2HcKcYBQoBmTfPY777x91Tk3v9N2gL0R0hEAF8T1AoCf2ekLc3cdwR/9zcu9C/PyV5aXBuNOu63m7r3v+GA8PTU5GMehfgGlmP8BS3ZOvKyI+CVmaUutG6/F4v76nzMS12HAz1pbW1XrJiYm+B5xrOYiEkQW6PuneTIYBzsIHQHxZLPR1PeP+J+0UqkMxkmSqHVp0hmMq5WqmiPxt81OjZ3bfidiT8Ms2guI6LeJ6A0ieqO+sXr9L3jcltjLiXQRwDFxfbT/mYJz7jkAzwHAfQ894sbHemwqcPrRnQazrzzRv7JKiX+5tSp/LzLnf4BsMC5H+jdSLQViHR/znSxT68oR/4pLsb6HPECiSJx+4iTsrZMsyrDpUmkwNocmGk1m73KqJL4DAA78vMCcarE4keRp2O101LpInGrVclnNgXZkrJ+JvZxIrwO4n4juIaISgK8CeHkP9/O4jXHDJ5JzLiWi3wXwbwBCAM87535Y2M48bivshbXBOfctAN8qaC8etzH2REi7BcEhop62JuUZACiFLEvEgZFbAtbwKnKd0ao6LZatwlDz/UrEmkm3w1phAK09upTnHOnXkwn1vBTz/aRM1Psi75+M9JDlLAc1m1oWXFleHowPHZjhexg5KCzxvkJjRwnFXqSIF5l7dITWHJn32O3qdzIMvIvEoxB4QvIoBKNlbeRQ6rOtPNUGshB85MeBVpljMRdkzA5KsWZfFDJLiQN9/zjgPzUnYfjLtVqctoUJIaypubYw6o2NMWsLDduAtCIbF1RDGFvffPMtNdcVrHlm8ku8j7L+vUtORM5YrHPefyAt5U6LC3nO7Mvles7lnrV53CR4QvIoBJ6QPArBiNV/Qqnv13DGDB8HQpbItNwSChWdxFwM7ZroCtU9y43KPMluBnIscyHXslSeCpkj0zJYfXN9MB4fY1dKYOQg6RCNYv2K14XKv7qp1f+qcOskQkxJuloOikr8PGdkpCzjvy0VcmiS6HdaEq4UZyIDcuM2Ggb+RPIoBJ6QPArBSFlbQA5l6h2bGWkVU6r80vIMaOuzy4VV2lieo2B7a21IfOQ7xToNWxKW50yYHQCgvrU5GJ+X1nETaSDZzbHJMTUnrdfff/ddNfcTDz88GOdi/51Ms9+KY69+blhzq8nXpYj3kXY1Gw0j3lc31f8WnY5eOwz8ieRRCDwheRSCkbI2uBxhX7PKzVEbCA2jtbGp5iCOWhcwuwmrevslwZZKkdboqNsYjDN5dGdmnYiWc6TZRqOxMRhfvsz3qE2Oq3VOxMm6SO8xqfP3KsYyv7zOWuFbP2C2VyvrPR6/997BODKsudPcGoyrEc/lHR1SnAntNNPRwEDbvP8h4E8kj0LgCcmjEHhC8igEo1X/AVT6gVdkrMFSRiobT/W4sFJPycD3jYZaVxZe7IqJNQuaLCMEbRFBEJjA94yflWxqGWmixmtnZmcH448XltS6sxf4+tRHr6i5tassB9XbRnXvcqRyCJ7rCtkMAH78wQcG41/55afU3JFDc4Nxp8Lvo93Q7ypp8B4nTdoatbawW/gTyaMQeELyKAQjZW1JkuDCJ58AALpdfaxvbfJxmnW1g/HiRU6XWyuzrtqoazX14Byzm/FaRc2FER/ziYhJjko6yzSI2LnbaGsTRVsGsIm8vPOXrqp1Hy9wImgj0TlplamDgzHVtLNUGhFqIg9v8dwpte7SpcuD8Wuv/beae+h+Ng3MT3NWcqu+rtY1NlcG4+5DD6q5+sYadgt/InkUAk9IHoXAE5JHIRipjFSv1/Ha/3wXAECkzf65UN1bLa2qfrJ0aTBW+ffmZzAjKpXUKlo2KYvvxcJ9Epm890Dk/jeNeh6J+zuRN7e0WlfrujlvbGxiWm9SRDJIdwkABKKgTLvN72ByYlKte+KnvzgYN0xhjrZILjh/nmWdM2fOqHWtlO0j51a0+6TV1O9/GFz3RCKi54noChH9QHw2S0TfJqLT/f/P7HQPj/2PYVjb3wF4ynz2DQCvOOfuB/BK/9rjDsZ1WZtz7j+J6PPm46cB/Hx//AKA/wDwh9e7V7Od4J3TZwEAY9UJNeccH/mdVB/5UzNsrS2XRPGotj6Cl+t8rIcmJnyiwjlqqYhrJlO6Jgz5/hTpvLZyg00PSZdND6urtu6TyCczKW+JCKrbamiWkrR47tg8mzLmZu5S62QUwuraspqbm+b9P/YIB8otLOqKQxstZu8fLqyouSDQYscwuFFh+5BzbrE/XgJw6Abv47FPsGetzfWqmW5b0VRWbEuS9nbLPG5z3KjWdpmIDjvnFonoMIAr2y2UFdtqU7Nuq68tuFzT8NgY23WrobZKHz1232DcTZgtLS9pZ+nVFT6iDx06qObKB44Oxo11Xpeb9PCpGT5cy2WtQ7RFCHczZdZWqWmtKuuyFheSrbrC2l5cMulUFb5+/KeYLT3wubv1PhJm6R+f0e/qzEmuBfvlL7F2d+yYvsf5d7k0ZDfT50CejS5l+2UAX+uPvwbgX27wPh77BMOo//8A4H8BPEhEC0T0dQDPAniSiE4D+MX+tccdjGG0tme2mfqFgvficRtjtCnbQYi43JOF5g9qnl0R3u6rVxfUXKMhAq1EkFvbVBabmmc1+cg9x9XcxBTLO5MHWH5aWdWe7iznV9I1mcvS4t5sshyUdLUaD5EPVyrpV1wps0khdtpyfnCSZa35GR5XjIliXshxkyUdub9y/vxgfO7MJ4PxXbMH1LqNy9/lfczqwLYk3D1ZeF+bRyHwhORRCEbK2sIwwvT0gcFYoiNSoG0Bz9UVDsra3BSqtckLC3NWn89dvKzmJjeZ/UxNsSM1NKaGjnDUkkkrL8vKIjVOea466/iVAXDavFCr8vdip1PCj84x2xsTpoHGpg5KSwVbtXVQ7xEs/YMPzw7GDzygg9dkC4/FS9rqXZ6ZxW7hTySPQuAJyaMQeELyKAQjrmpLA7mm2TJVbQWzDyMtt2SZaJEVsSsld6aVVpkjCg4cOKzmxsc5yL9S5ftPlfWzopgD4mxVOSdcCanoLzdlcv8DUX0uNyVpIqHy5x0dEDclou9cypEAmalgl6QsP7VM8N3YxNRgfG6JXUHvn/l3ta4jagF0O6ZofTg677+Hh4InJI9CMFLWFkUx5vrW59wU2ByviipkmbYUxwGzn4PCIk6RtuqWRCfEkmFZlYro3yGCvS37IlUN3VZ9k50bmS0FRsWXZgIXaP28ucHs5uInp9XcaszPmxYlew7N6bjvSoVNCO3EsKWITRHRGFvHlxcuqXXHDrM1eyLR+9/s+ILtHjcJnpA8CsFoq5EEIcb6x23XaBtVkWI9PamD0nKROhOJtpzVcRP3LVKcAmM5z52Yk78f81OSiqDtwJ2KRsqp6ImyuaJTtuWTY8Pa6hscY714SbObQ7PMiqZr7GRtGtaTC9acmn9CqVkeOcqdYh8UqdwA8OgJvj519oKae/u9D7Bb+BPJoxB4QvIoBJ6QPArBSGWk3OVotHpe/gmT1yajAa4s6zyrzQ32fuciaeC48WhPi+Ct0LRQJ1HpLc1Y5rA9OpoisL5tCpenCQf8k8iNc6YVek0Em01Pa096tcRqd2Rc99PjrNZPTfA4MfdvineQdHQEQSAiFmQK+5jp+bZwgYP/QxNB8PCD92O38CeSRyHwhORRCEbutC3HvWN/5apOhTuzxiq0dVJOz3C89eHDHK+cpPpY74oEzNwUNN0UFTZaouVnlpqWXkJdL5lYacmyKjW2oldNK622CDzLjQmhNs4OXptWXhLO0jDkZ8cmLrsteodQaFm46OkiquItrOjY9KZI+44iHZh31+Gj2C38ieRRCDwheRQCT0gehWCkMlKWplhf66n2ixe1e2BMBNN/4cQX1dysyEOTbdLbprLb2hqXl+mayrhNJ1uti8C2SS0f1EQFt6qRTSIh02zX8rP3bFEoPdCedBJV2Wz5mEy4N7pCJY9CXX1O9qxrm952K8ssa14VrputLV2EfU000KmNmfI9E3PYLYZJ2T5GRK8S0ftE9EMi+r3+575qm8cAw7C2FMAfOOdOAHgCwO8Q0Qn4qm0eAsPk/i8CWOyPt4joAwBHcANV26Ioxux8T32fOaA9/JFQY6OKDkrbqrM6XRdF2stlzXqkupsb08Ddh9iiXBaFSkPjnXc5s6JGWwfYtUVR+XXBRldWddU0mdr9kCmGHk9zkJop5oZQVFqVKn6nodnSwhJ765ev6mcnCb+Dpug/srGu+5mUhCdBvl8AeOU738FusSthu18C8CcBfA++apuHwNCERETjAP4JwO8751Tvhp2qtsmKba3G7rvueNweGIqQiChGj4j+3jn3z/2PL/ertWGnqm3Oueecc4855x6r1iY+a4nHPsB1ZSQiIgB/C+AD59xfiKlPq7Y9iyGrtjkA3X6ftkpFq92RCOTPTDC9zHmLhOsgMEJGRcg+rYbphbbBp6FsRxaV9G8pEG4RZ0rgnfyAy+qd7zfnAYDU5K454Z65+7CuSDs7xXlnraaOLpDX62usnq+s6WiIVsKyW2b22BT32NhkxhEYhjEmeu0uLS6quSVTUnEYDGNH+lkAvwngPSJ6p//ZN9EjoJf6FdzOAfj1XT/dY99gGK3tv3CtgvEpfNU2DwAjtmy3O22cPtULLD/x8Ak1VxVsKdecTfXokD1LLl/RYlljk1XcTsu2J2cWINnBvcc/r9bNH+TguMxsJBbsd0oEjZVN3xPpkLeW5w9PnhyM6w2tdsu1XbHf3LRtbQgrdcv8nU0R5SBNAWXTNn7zClu919d12Zws37ba9bbwvjaPQuAJyaMQjJS1uTxDt907ltumNWYgNB9nNAyZo5YJi/Xp07pFZ13EdpdMsFksUrilFT1PdQBckAp2ZgqZz4nO2lJjbLY0i2qJ6wsXdGFV+T26JqdOpISLIL0Nw3oaK8zCY8OyZJWUNOO/rbGu27amwvqeZabq6vaNHLaFP5E8CoEnJI9C4AnJoxCMNvefgEo/bz0xckVFVIIlE/AVSGu2kH0mTaW0ishlGxeBcgAQCkv6mIguSLs6SuD0hx8OxhumD9uG8BVmwnptm9NI63u5pE0DJKINmia6YHmVLdhNYQoIzfuYmeQIgqStzQtSXktFgF1+jRy0ffkessLbEPAnkkch8ITkUQhGytoAGsQpZ6m2Gsuu21Yl73QESxHqbdWovkHMludWQ8dzd1Y5RvyCzDtLTUy1sCLHsQ6ck0VSY9FbLTBvMRFV1OprJjiuXRdj7bSVDKYiTAG2BFAXship6ZAtLN25sMyT8XCngl26TJ8npXg7j9j28CeSRyHwhORRCDwheRSC0ea1ZSm21nte59aWNvtfucRqcqetc9Jkfr4M8O+afm1OyDeBkQnimOWuSJTOC03ufCRMCLbVumzz3haBc52Olse2RAMdZwrE1iZYzgoD/Tt2Qm7sNFh+Sk0iw0aHn229/5mIjpA5dLkJFpSITHVgyn1VW4+bBE9IHoVgpKwtTdpYOtcrUu5M0Jj0QFtVNRL5a7KgOtmyMKKPyNiYtmzLtVItTo36X68zG0lMMfTciXRrklZjzXpKZX72wbt1y9VGnT33m+u61EwqWtE7WbrGBKg2E8n2tmfvynht7hGLdxya0jvN5u6zffyJ5FEIPCF5FILRWradQ5j3tIw8M0XI5VFuWFsmTMeB47HVqjqi0lva1VZjyZauDeRiRMJaHhuHayi0m0iwEGulr4jO2uWqTrtaW+E9Nra04zoW1mbZ98QWI02FBmaDACULD4RWSCbuuxLxs+qm1ams5jYs/InkUQg8IXkUAk9IHoVgxN5/N1CVneHZTpiAXW5agIryZcpMYO5OQibIjMU6FKaBsqjKZq3LgfieDYF3wmqciYpwmbEuJ6KNfMtUlWvUd4g8EAFybZF6fc27Uo13NKSMJOcia0UXherXVnRr+26i/55hMEzFtgoR/R8Rfb9fse1P+p/fQ0TfI6KPiOgfiah0vXt57F8Mw9o6AL7inHsEwKMAniKiJwD8OYC/dM4dB7AG4Os/um163OoYJvffAfj0PI77/zkAXwHwG/3PXwDwxwD+eqd75XmOdj+NODJBabIlaGgD1oTaLXPcAnNcy9jmwHaKFnHUkgVaC7u0FEsHKAB0RcBdKALKunVtCc7EPmomZVuys8DYLzotsXaHtGmbwi0h9x+JwDzrnF69zOnuXeN0tmaVYTBsfaSwX4nkCoBvAzgDYN2xYLOAXjlAjzsUQxGScy5zzj0K4CiAxwF8YdgHyIpteb69IdDj9sau1H/n3DqAVwF8GcA0EX3KZ44CuLjNdwYV22xdaY/9g2Eqts0D6Drn1omoCuBJ9ATtVwH8GoAXMWTFNgqCQQ6+JapYyC1W9nHCXSDZN1lRQcgVzkaUCbOBLFdjEw1knpssCwPoQPtMNMZJjfpfE/esTuni59LDb4P6rcz0KWyUg6z7Y8oTKJdJTciFjU0dabAp3SLmHoHKZhiOiwxjRzoM4AXqpXkEAF5yzv0rEb0P4EUi+lMAb6NXHtDjDsUwWtu76JVEtp+fRU9e8vAAWavpj/RhRMvo1Zs8AODqdZbfKbjV38XnnHPz11s0UkIaPJToDefcYyN/8C2I/fIuvNPWoxB4QvIoBDeLkJ67Sc+9FbEv3sVNkZE89h88a/MoBCMlJCJ6iohO9mOY7rhGgfu5G+fIWFvfMn4KPRfLAoDXATzjnHt/xy/uI/S7SB12zr1FRBMA3gTwqwB+C8Cqc+7Z/g9sxjm3YxPFWw2jPJEeB/CRc+6scy5Bz0f39Aiff9PhnFt0zr3VH28BkN04X+gvewE94rqtMEpCOgLggri+o2OY9ls3Ti9s3wTcaDfOWxmjJKSLAI6J621jmPYz9tKN81bGKAnpdQD397NPSgC+il4XyjsGQ3TjBIaM7brVMGrv/y8B+CsAIYDnnXN/NrKH3wIgop8D8BqA94BBLZlvoicnvQTgx9DvxumcW/3Mm9yi8JZtj0LghW2PQuAJyaMQeELyKASekDwKgSckj0LgCcmjEHhC8igEnpA8CsH/A/Tu8iGj/sh6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 144x144 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#获取测试集中的一个batch的数据\n",
    "X, label = next(test_loader())\n",
    "logits = runner.predict(X)\n",
    "#多分类，使用softmax计算预测概率\n",
    "pred = F.softmax(logits)\n",
    "#获取概率最大的类别\n",
    "pred_class = paddle.argmax(pred[2]).numpy()\n",
    "label = label[2][0].numpy()\n",
    "#输出真实类别与预测类别\n",
    "print(\"The true category is {} and the predicted category is {}\".format(label[0], pred_class[0]))\n",
    "#可视化图片\n",
    "plt.figure(figsize=(2, 2))\n",
    "imgs, labels = load_cifar10_batch(folder_path='/home/aistudio/datasets/cifar-10-batches-py', mode='test')\n",
    "plt.imshow(imgs[2].transpose(1,2,0))\n",
    "plt.savefig('cnn-test-vis.pdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 5.6 实验拓展\n",
    "\n",
    "* 尝试加深残差网络的层数或使用《神经网络与深度学习》中介绍的其他模型完成基于CIFAR10的图像分类实验，观察是否能够得到更高的精度；\n",
    "* 尝试使用CIFAR-100进行实验，观察不同模型在不同数据集上的学习效果；"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
